vitalif
/
qperf
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Initial commit

master
Johann George (QLogic) 2007-09-24 11:11:24 -07:00
commit 529b2a4458
9 changed files with 7530 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

339
COPYING Normal file
View File

@ -0,0 +1,339 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.

18
Makefile Normal file
View File

@ -0,0 +1,18 @@
CC = gcc
CFLAGS = -Wall -O -DRDMA
all: qperf
qperf: qperf.o ip.o ib.o help.o
$(CC) -DRDMA -o $@ $^ -libverbs
help.c: help.txt
./mkhelp RDMA
.PHONY: clean
clean:
rm -f *.o help.c qperf
.PHONY: install
install:
cp qperf /usr/local/bin

55
configure vendored Executable file
View File

@ -0,0 +1,55 @@
#!/bin/sh
#
LIBIBV=libibverbs.so.1
# Look for InfiniBand verbs library.
#
for DIR in /lib /usr/lib64 /usr/lib /usr/local/lib /usr/local/ibed/lib64 ""; do
[ -e $DIR/$LIBIBV ] &&
break
done
# Set up parameters.
#
if [ "`uname -m`" = ppc64 ]; then
K_M64=-m64
echo "Making PowerPC version"
fi
if [ -z "$DIR" ]; then
echo "Could not find $LIBIBV; making non-RDMA version"
else
echo "Found $LIBIBV in $DIR"
K_IB_O="ib.o"
K_DEF_IB="-DRDMA"
if [ "$DIR" = "/usr/lib64" ]; then
K_LIBS="-libverbs"
else
K_LIBS="-lsysfs -Wl,--rpath -Wl,$DIR $DIR/$LIBIBV"
fi
K_IB=RDMA
fi
# Produce Makefile
#
cat <<EOF >Makefile
CC = gcc $K_M64
CFLAGS = -Wall -O $K_DEF_IB
all: qperf
qperf: qperf.o ip.o $K_IB_O help.o
\$(CC) $K_DEF_IB -o \$@ \$^ $K_LIBS
help.c: help.txt
./mkhelp $K_IB
.PHONY: clean
clean:
rm -f *.o help.c qperf
.PHONY: install
install:
cp qperf /usr/local/bin
EOF

708
help.txt Normal file
View File

@ -0,0 +1,708 @@
Main
Usage:
qperf [options] <server_host> <test1> <test2> ...
Description
One typically runs qperf with no arguments on the server machine. On a
client machine, one specifies the hostname of the server machine
followed by a list of tests that might be run.
More Information
qperf --help examples Some examples of using qperf
qperf --help opts Summary of options
qperf --help options Description of options
qperf --help tests Short summary and description of tests
qperf --help <test_name> More information on a particular test
Opts
--access_recv Mode (-Ar) Access receive data
--affinity PN (-a) Set processor affinity
--loc_affinity PN (-la) Set local processor affinity
--rem_affinity PN (-ra) Set remote processor affinity
--flip Mode (-f) Flip sender and receiver
--help Topic (-h) Get more information on a topic
--host Node (-H) Identify server node
--id Device:Port (-i) Set IB device and port
--loc_id Device:Port (-li) Set local IB device and port
--rem_id Device:Port (-ri) Set remote IB device and port
--listen_port Port (-lp) Set server listen port
--msg_size Size (-m) Set message size
--mtu_size Size (-M) Set MTU size (IB only)
--no_msgs Count (-n) Send Count messages
--poll Mode (-P) Set polling mode on/off
--loc_poll Mode (-lP) Set local polling mode on/off
--rem_poll Mode (-lP) Set remote polling mode on/off
--port Port (-p) Set TCP port used for tests
--precision Digits (-e) Set precision reported
--rate (-r) Set IB static rate
--loc_rate (-lr) Set local IB static rate
--rem_rate (-rr) Set remote IB static rate
--rd_atomic Count (-R) Set RDMA read/atomic count
--loc_rd_atomic Count (-lR) Set local RDMA read/atomic count
--rem_rd_atomic Count (-rR) Set remote RDMA read/atomic count
--sock_buf_size Size (-S) Set socket buffer size
--loc_sock_buf_size Size (-lS) Set local socket buffer size
--rem_sock_buf_size Size (-rS) Set remote socket buffer size
--time (-t) Set test duration
--timeout Time (-T) Set timeout
--loc_timeout Time (-lT) Set local timeout
--rem_timeout Time (-rT) Set remote timeout
--unify_nodes (-U) Unify nodes
--unify_units (-u) Unify units
--verbose (-v) Verbose; turn on all of -v[cstu]
--verbose_conf (-vc) Show configuration information
--verbose_stat (-vs) Show statistical information
--verbose_time (-vt) Show timing information
--verbose_used (-vu) Show information on parameters
--verbose_more (-vv) More verbose; turn on all of -v[CSTU]
--verbose_more_conf (-vC) Show more configuration information
--verbose_more_stat (-vS) Show more statistical information
--verbose_more_time (-vT) Show more timing information
--verbose_more_used (-vU) Show more information on parameters
--version (-V) Print out version
--wait Time (-W) Set time to wait for server
Options
--access_recv Mode (-Ar)
If Mode is non-zero, data is accessed once received. Otherwise, data
is ignored. By default, Mode is 0.
--affinity PN (-a)
Set processor affinity to PN. Processors are numbered sequentially
from 0. If PN is "any", any processor is allowed otherwise the
processor is limited to the one specified.
--loc_affinity PN (-la)
Set local processor affinity to PN.
--rem_affinity PN (-ra)
Set remote processor affinity to PN.
--flip Mode (-f)
Cause sender and receiver to play opposite roles.
--help Topic (-h)
Print out information about Topic. To see the list of topics, type
qperf --help
--host Host (-H)
Run test between the current node and the qperf running on node Host.
This can also be specified as the first non-option argument.
--id Device:Port (-i)
Use InfiniBand Device and Port.
--loc_id Device:Port (-li)
Use local InfiniBand Device and Port.
--rem_id Device:Port (-ri)
Use remote InfiniBand Device and Port.
--listen_port Port (-lp)
Set the port we listen on to ListenPort. This must be set to the
same port on both the server and client machines. The default value
is 19765.
--msg_size Size (-m)
Set the message size to Size. The default value varies by test. It
is assumed that the value is specified in bytes however, a trailing
kib or K, mib or M, or gib or G indicates that the size is being
specified in kibibytes, mebibytes or gibibytes respectively while a
trailing kb or k, mb or m, or gb or g indicates kilobytes, megabytes
or gigabytes respectively.
--mtu_size Size (-M)
Set the MTU size. Only relevant to the RDMA UC/RC tests. Units are
specified in the same manner as the --msg_size option.
--no_msgs N (-n)
Set test duration by number of messages sent instead of time.
--poll Mode (-P)
Turn polling mode on or off. This is only relevant to the RDMA tests
and determines whether they poll or wait. If Mode is 0, they wait;
otherwise they poll.
--loc_poll Mode (-lP)
Locally turn polling mode on or off.
--rem_poll Mode (-rP)
Remotely turn polling mode on or off.
--port Port (-p)
Use Port to run the socket tests. This is different from
--listen_port which is used for synchronization. This is only
relevant for the socket tests and refers to the TCP/UDP/SDP/RDS port
that the test is run on.
--precision Digits (-e)
Set the number of significant digits that are used to report results.
--rate Rate (-r)
Force InfiniBand static rate. Rate can be one of: 2.5, 5, 10, 20,
30, 40, 60, 80, 120, 1xSDR (2.5 Gbps), 1xDDR (5 Gbps), 1xQDR (10
Gbps), 4xSDR (2.5 Gbps), 4xDDR (5 Gbps), 4xQDR (10 Gbps), 8xSDR (2.5
Gbps), 8xDDR (5 Gbps), 8xQDR (10 Gbps).
--loc_rate (-lr)
Force local InfiniBand static rate
--rem_rate (-rr)
Force remote InfiniBand static rate
--rd_atomic Count (-R)
Set the number of in-flight operations that can be handled for a RDMA
read or atomic operation. This is only relevant to the RDMA tests.
--loc_rd_atomic Count (-lR)
Set local read/atomic count.
--rem_rd_atomic Count (-rR)
Set remote read/atomic count.
--sock_buf_size Size (-S)
Set the socket buffer size. This is only relevant to the socket
tests.
--loc_sock_buf_size Size (-lS)
Set local socket buffer size.
--rem_sock_buf_size Size (-rS)
Set remote socket buffer size.
--time Time (-t)
Set test duration to Time. Specified in seconds however a trailing
m, h or d indicates that the time is specified in minutes, hours or
days respectively.
--timeout Time (-T)
Set timeout to Time. This is the timeout used for various things
such as exchanging messages. The default is 5 seconds.
--loc_timeout Time (-lT)
Set local timeout to Time.
--rem_timeout Time (-rT)
Set local timeout to Time.
--unify_nodes (-U)
Unify the nodes. Describe them in terms of local and remote rather
than send and receive.
--unify_units (-u)
Unify the units that results are shown in. Uses the lowest common
denominator. Helpful for scripts.
--verbose (-v)
Provide more detailed output. Turns on -vc, -vs, -vt and -vu.
--verbose_conf (-vc)
Provide information on configuration.
--verbose_stat (-vs)
Provide information on statistics.
--verbose_stat (-vt)
Provide information on timing.
--verbose_stat (-vu)
Provide information on parameters used.
--verbose_more (-vv)
Provide even more detailed output. Turns on -vC, -vS, -vT and -vU.
--verbose_conf (-vC)
Provide more information on configuration.
--verbose_stat (-vS)
Provide more information on statistics.
--verbose_stat (-vT)
Provide more information on timing.
--verbose_stat (-vU)
Provide more information on parameters used.
--version (-V)
The current version of qperf is printed.
--wait Time (-W)
If the server is not ready, continue to try connecting for Time
seconds before giving up.
Examples
For these examples, we assume that qperf is running on a machine called
myserver in server mode. To run qperf in server mode, run it with no
arguments. In all the subsequent examples, we run qperf on another machine
and connect to the server which we assume has a hostname of myserver.
* To run a TCP bandwidth and latency test:
qperf myserver tcp_bw tcp_lat
* To run a SDP bandwidth test for 10 seconds:
qperf -t 10 myserver sdp_bw
* To run a UDP latency test and then cause the server to terminate:
qperf myserver udp_lat quit
* To measure the RDMA UD latency and bandwidth:
qperf myserver ud_lat ud_bw
* To measure RDMA UC bi-directional bandwidth:
qperf myserver rc_bi_bw
Tests -RDMA
Miscellaneous
conf Show configuration
quit Cause the server to quit
Socket Based
tcp_bw TCP streaming one way bandwidth
tcp_lat TCP one way latency
udp_bw UDP streaming one way bandwidth
udp_lat UDP one way latency
sdp_bw SDP streaming one way bandwidth
sdp_lat SDP one way latency
rds_bw RDS streaming one way bandwidth
rds_lat RDS one way latency
Tests +RDMA
Miscellaneous
conf Show configuration
quit Cause the server to quit
Socket Based
tcp_bw TCP streaming one way bandwidth
tcp_lat TCP one way latency
udp_bw UDP streaming one way bandwidth
udp_lat UDP one way latency
sdp_bw SDP streaming one way bandwidth
sdp_lat SDP one way latency
rds_bw RDS streaming one way bandwidth
rds_lat RDS one way latency
RDMA Send/Receive
ud_bw UD streaming one way bandwidth
ud_bi_bw UD streaming two way bandwidth
ud_lat UD one way latency
rc_bw RC streaming one way bandwidth
rc_bi_bw RC streaming two way bandwidth
rc_lat RC one way latency
uc_bw UC streaming one way bandwidth
uc_bi_bw UC streaming two way bandwidth
uc_lat UC one way latency
RDMA
rc_rdma_read_bw RC RDMA read streaming one way bandwidth
rc_rdma_read_lat RC RDMA read one way latency
rc_rdma_write_bw RC RDMA write streaming one way bandwidth
rc_rdma_write_lat RC RDMA write one way latency
rc_rdma_write_poll_lat RC RDMA write one way polling latency
uc_rdma_write_bw UC RDMA write streaming one way bandwidth
uc_rdma_write_lat UC RDMA write one way latency
uc_rdma_write_poll_lat UC RDMA write one way polling latency
InfiniBand Atomics
rc_compare_swap_mr RC compare and swap messaging rate
rc_fetch_add_mr RC fetch and add messaging rate
Verification
ver_rc_compare_swap verify RC compare and swap
ver_rc_fetch_add verify RC fetch and add
conf
Purpose
Show configuration
Common Options
None
Description
Shows the node name, CPUs and OS of both nodes being used.
quit
Purpose
Quit
Common Options
None
Description
Causes the server to quit.
tcp_bw
Purpose
TCP streaming one way bandwidth
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly sends messages to the server while the server
notes how many were received.
tcp_lat
Purpose
TCP one way latency
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using TCP sockets.
udp_bw
Purpose
UDP streaming one way bandwidth
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly sends messages to the server while the server
notes how many were received.
udp_lat
Purpose
UDP one way latency
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using UDP sockets.
sdp_bw
Purpose
SDP streaming one way bandwidth
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly sends messages to the server while the server
notes how many were received.
sdp_lat
Purpose
SDP one way latency
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using SDP sockets.
rds_bw
Purpose
RDS streaming one way bandwidth
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly sends messages to the server while the server
notes how many were received.
rds_lat
Purpose
RDS one way latency
Common Options
--affinity PN (-a) set processor affinity
--msg_size Size (-m) set message size
--sock_buf_size Size (-S) set socket buffer size
--time (-t) set test duration
Other Options
--listen_port, --port, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using RDS sockets.
ud_bw +RDMA
Purpose
UD streaming one way bandwidth
Common Options
--access_recv OnOff (-Ar) Access receive data
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client sends messages to the server who notes how many it received.
The UD Send/Receive mechanism is used.
ud_bi_bw +RDMA
Purpose
UD streaming two way bandwidth
Common Options
--access_recv OnOff (-Ar) Access receive data
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
Both the client and server exchange messages with each other using the
UD Send/Receive mechanism and note how many were received.
ud_lat +RDMA
Purpose
UD one way latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using UD Send/Receive.
rc_bw +RDMA
Purpose
RC streaming one way bandwidth
Common Options
--access_recv OnOff (-Ar) Access receive data
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client sends messages to the server who notes how many it received.
The RC Send/Receive mechanism is used.
rc_bi_bw +RDMA
Purpose
RC streaming two way bandwidth
Common Options
--access_recv OnOff (-Ar) Access receive data
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
Both the client and server exchange messages with each other using the
RC Send/Receive mechanism and note how many were received.
rc_lat +RDMA
Purpose
RC one way latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using RC Send/Receive.
uc_bw +RDMA
Purpose
UC streaming one way bandwidth
Common Options
--access_recv OnOff (-Ar) Access receive data
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client sends messages to the server who notes how many it received.
The UC Send/Receive mechanism is used.
uc_bi_bw +RDMA
Purpose
UC streaming two way bandwidth
Common Options
--access_recv OnOff (-Ar) Access receive data
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
Both the client and server exchange messages with each other using the
UC Send/Receive mechanism and note how many were received.
uc_lat +RDMA
Purpose
UC one way latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
repeatedly using UC Send/Receive.
rc_rdma_read_bw +RDMA
Purpose
RC RDMA read streaming one way bandwidth
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --rd_atomic, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly performs RC RDMA Read operations and notes how
many of them complete.
rc_rdma_read_lat +RDMA
Purpose
RC RDMA read one way latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly performs RC RDMA Read operations waiting for
completion before starting the next one.
rc_rdma_write_bw +RDMA
Purpose
RC RDMA write streaming one way bandwidth
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly performs RC RDMA Write operations and notes how
many of them complete.
rc_rdma_write_lat +RDMA
Purpose
RC RDMA write one way latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
using RC RDMA write operations.
rc_rdma_write_poll_lat +RDMA
Purpose
RC RDMA write one way polling latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test using RC RDMA Write operations. First the
client performs an RDMA Write while the server stays in a tight loop
waiting for the memory buffer to change. The first and last bytes of
the memory buffer are tested to ensure that the entire message was
received. This is then repeated with both sides playing opposite
roles. Since this always polls, the -P (--poll) flag has no effect.
uc_rdma_write_bw +RDMA
Purpose
UC RDMA write streaming one way bandwidth
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly performs UC RDMA Write operations and notes how
many of them complete.
uc_rdma_write_lat +RDMA
Purpose
UC RDMA write one way latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test where the server and client exchange messages
using UC RDMA write operations.
uc_rdma_write_poll_lat +RDMA
Purpose
UC RDMA write one way polling latency
Common Options
--id Device:Port (-i) Set IB device and port
--msg_size Size (-m) Set message size
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
A ping pong latency test using UC RDMA Write operations. First the
client performs an RDMA Write while the server stays in a tight loop
waiting for the memory buffer to change. The first and last bytes of
the memory buffer are tested to ensure that the entire message was
received. This is then repeated with both sides playing opposite
roles. Since this always polls, the -P (--poll) flag has no effect.
rc_compare_swap_mr +RDMA
Purpose
RC compare and swap messaging rate
Common Options
--id Device:Port (-i) Set IB device and port
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --rd_atomic, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly performs the RC Atomic Compare and Swap operation
and determines how many of them complete.
rc_fetch_add_mr +RDMA
Purpose
RC fetch and add messaging rate
Common Options
--id Device:Port (-i) Set IB device and port
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --rd_atomic, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
The client repeatedly performs the RC Atomic Fetch and Add operation
and determines how many of them complete.
ver_rc_compare_swap +RDMA
Purpose
Verify RC compare and swap
Common Options
--id Device:Port (-i) Set IB device and port
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --rd_atomic, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
Test the RC Compare and Swap Atomic operation. The server's memory
location starts with zero and the client successively exchanges, 0 for
1, 1 for 2, etc. The results are checked for correctness.
ver_rc_fetch_add +RDMA
Purpose
Verify RC fetch and add
Common Options
--affinity PN (-a) Set processor affinity
--id Device:Port (-i) Set IB device and port
--poll OnOff (-P) Set polling mode on/off
--time (-t) Set test duration
Other Options
--affinity, --listen_port, --mtu_size, --rate, --rd_atomic, --timeout
Display Options
--precision, --unify_nodes, --unify_units, --verbose
Description
Tests the RC Fetch and Add Atomic operation. The server's memory
location starts with zero and the client successively adds one. The
results are checked for correctness.

2231
ib.c Normal file
View File

File diff suppressed because it is too large Load Diff

837
ip.c Normal file
View File

@ -0,0 +1,837 @@
/*
* qperf - handle socket tests.
*
* Copyright (c) 2002-2007 Johann George. All rights reserved.
* Copyright (c) 2006-2007 QLogic Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
* General Public License (GPL) Version 2, available from the file
* COPYING in the main directory of this source tree, or the
* OpenIB.org BSD license below:
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above
* copyright notice, this list of conditions and the following
* disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials
* provided with the distribution.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#define _GNU_SOURCE
#include <netdb.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "qperf.h"
/*
* Parameters.
*/
#define AF_INET_SDP 27 /* Family for SDP */
#define AF_INET_RDS 30 /* Family for RDS */
/*
* Function prototypes.
*/
static void datagram_client_bw(int domain);
static void datagram_client_init(int *fd, int domain,
struct sockaddr_in *addr);
static void datagram_client_lat(int domain);
static void datagram_server_bw(int domain);
static int datagram_server_init(int *fd, int domain);
static void datagram_server_lat(int domain);
static uint32_t decode_port(uint32_t *p);
static void encode_port(uint32_t *p, uint32_t port);
static void ip_parameters(long msgSize);
static int recv_full(int fd, void *ptr, int len);
static int send_full(int fd, void *ptr, int len);
static int set_socket_buffer_size(int fd);
static void socket_client_bw(int domain);
static void socket_client_init(int *fd, int domain);
static void socket_client_lat(int domain);
static void socket_server_bw(int domain);
static int socket_server_init(int *fd, int domain);
static void socket_server_lat(int domain);
/*
* Measure RDS bandwidth (client side).
*/
void
run_client_rds_bw(void)
{
ip_parameters(8*1024);
datagram_client_bw(AF_INET_RDS);
}
/*
* Measure RDS bandwidth (server side).
*/
void
run_server_rds_bw(void)
{
datagram_server_bw(AF_INET_RDS);
}
/*
* Measure RDS latency (client side).
*/
void
run_client_rds_lat(void)
{
ip_parameters(1);
datagram_client_lat(AF_INET_RDS);
}
/*
* Measure RDS latency (server side).
*/
void
run_server_rds_lat(void)
{
datagram_server_lat(AF_INET_RDS);
}
/*
* Measure UDP bandwidth (client side).
*/
void
run_client_udp_bw(void)
{
ip_parameters(32*1024);
datagram_client_bw(AF_INET);
}
/*
* Measure UDP bandwidth (server side).
*/
void
run_server_udp_bw(void)
{
datagram_server_bw(AF_INET);
}
/*
* Measure UDP latency (client side).
*/
void
run_client_udp_lat(void)
{
ip_parameters(1);
datagram_client_lat(AF_INET);
}
/*
* Measure UDP latency (server side).
*/
void
run_server_udp_lat(void)
{
datagram_server_lat(AF_INET);
}
/*
* Measure SDP bandwidth (client side).
*/
void
run_client_sdp_bw(void)
{
ip_parameters(64*1024);
socket_client_bw(AF_INET_SDP);
}
/*
* Measure SDP bandwidth (server side).
*/
void
run_server_sdp_bw(void)
{
socket_server_bw(AF_INET_SDP);
}
/*
* Measure SDP latency (client side).
*/
void
run_client_sdp_lat(void)
{
ip_parameters(1);
socket_client_lat(AF_INET_SDP);
}
/*
* Measure SDP latency (server side).
*/
void
run_server_sdp_lat(void)
{
socket_server_lat(AF_INET_SDP);
}
/*
* Measure TCP bandwidth (client side).
*/
void
run_client_tcp_bw(void)
{
ip_parameters(64*1024);
socket_client_bw(AF_INET);
}
/*
* Measure TCP bandwidth (server side).
*/
void
run_server_tcp_bw(void)
{
socket_server_bw(AF_INET);
}
/*
* Measure TCP latency (client side).
*/
void
run_client_tcp_lat(void)
{
ip_parameters(1);
socket_client_lat(AF_INET);
}
/*
* Measure TCP latency (server side).
*/
void
run_server_tcp_lat(void)
{
socket_server_lat(AF_INET);
}
/*
* Measure socket bandwidth (client side).
*/
static void
socket_client_bw(int domain)
{
char *buf;
int sockFD;
socket_client_init(&sockFD, domain);
buf = qmalloc(Req.msg_size);
if (!synchronize())
goto err;
while (!Finished) {
int n = send_full(sockFD, buf, Req.msg_size);
if (Finished)
break;
if (n < 0) {
LStat.s.no_errs++;
continue;
} else {
LStat.s.no_bytes += n;
LStat.s.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
show_results(BANDWIDTH);
}
/*
* Measure socket bandwidth (server side).
*/
static void
socket_server_bw(int domain)
{
int sockFD;
char *buf = 0;
if (!socket_server_init(&sockFD, domain))
return;
if (!synchronize())
goto err;
buf = qmalloc(Req.msg_size);
while (!Finished) {
int n = recv_full(sockFD, buf, Req.msg_size);
if (Finished)
break;
if (n < 0) {
LStat.r.no_errs++;
continue;
} else {
LStat.r.no_bytes += n;
LStat.r.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
}
/*
* Measure socket latency (client side).
*/
static void
socket_client_lat(int domain)
{
char *buf;
int sockFD;
socket_client_init(&sockFD, domain);
buf = qmalloc(Req.msg_size);
if (!synchronize())
goto err;
while (!Finished) {
int n = send_full(sockFD, buf, Req.msg_size);
if (Finished)
break;
if (n < 0) {
LStat.s.no_errs++;
continue;
} else {
LStat.s.no_bytes += n;
LStat.s.no_msgs++;
}
n = recv_full(sockFD, buf, Req.msg_size);
if (Finished)
break;
if (n < 0) {
LStat.r.no_errs++;
continue;
} else {
LStat.r.no_bytes += n;
LStat.r.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
show_results(LATENCY);
}
/*
* Measure socket latency (server side).
*/
static void
socket_server_lat(int domain)
{
int sockFD;
char *buf = 0;
if (!socket_server_init(&sockFD, domain))
return;
if (!synchronize())
goto err;
buf = qmalloc(Req.msg_size);
while (!Finished) {
int n = recv_full(sockFD, buf, Req.msg_size);
if (Finished)
break;
if (n < 0) {
LStat.r.no_errs++;
continue;
} else {
LStat.r.no_bytes += n;
LStat.r.no_msgs++;
}
n = send_full(sockFD, buf, Req.msg_size);
if (Finished)
break;
if (n < 0) {
LStat.s.no_errs++;
continue;
} else {
LStat.s.no_bytes += n;
LStat.s.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
}
/*
* Socket client initialization.
*/
static void
socket_client_init(int *fd, int domain)
{
uint32_t port;
struct hostent *host;
struct sockaddr_in clientAddr;
struct sockaddr_in serverAddr;
socklen_t clientLen = sizeof(clientAddr);
client_send_request();
*fd = socket(domain, SOCK_STREAM, 0);
if (*fd < 0)
syserror_die("socket failed");
clientAddr.sin_family = AF_INET;
clientAddr.sin_addr.s_addr = htonl(INADDR_ANY);
clientAddr.sin_port = htons(0);
if (bind(*fd, (struct sockaddr *)&clientAddr, clientLen) < 0)
syserror_die("bind failed");
if (getsockname(*fd, (struct sockaddr *)&clientAddr, &clientLen) < 0)
syserror_die("getsockname failed");
if (!set_socket_buffer_size(*fd))
die();
host = gethostbyname(ServerName);
if (!host)
error_die("cannot find machine %s", ServerName);
serverAddr.sin_family = AF_INET;
if (host->h_length > sizeof(serverAddr.sin_addr))
error_die("address too large to handle");
memcpy(&serverAddr.sin_addr.s_addr, host->h_addr, host->h_length);
if (!recv_mesg(&port, sizeof(port), "port"))
die();
port = decode_port(&port);
debug("sending from %s port %d to %d",
domain == AF_INET_SDP ? "SDP" : "TCP",
ntohs(clientAddr.sin_port), port);
serverAddr.sin_port = htons(port);
if (connect(*fd, &serverAddr, sizeof(serverAddr)) < 0)
syserror_die("connect failed");
}
/*
* Socket server initialization.
*/
static int
socket_server_init(int *fd, int domain)
{
uint32_t port;
int listenFD;
struct sockaddr_in addr;
socklen_t len = sizeof(addr);
int stat = 0;
int one = 1;
listenFD = socket(domain, SOCK_STREAM, 0);
if (listenFD < 0)
return syserror("socket failed");
if (setsockopt(listenFD, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)) < 0)
return syserror("failed to reuse address on socket");
memset(&addr, 0, len);
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = htonl(INADDR_ANY);
addr.sin_port = htons(Req.port);
if (bind(listenFD, (struct sockaddr *)&addr, len) < 0) {
syserror("bind failed");
goto err;
}
if (getsockname(listenFD, (struct sockaddr *)&addr, &len) < 0) {
syserror("getsockname failed");
goto err;
}
port = ntohs(addr.sin_port);
if (listen(listenFD, 1) < 0) {
syserror("listen failed");
goto err;
}
encode_port(&port, port);
if (!send_mesg(&port, sizeof(port), "port"))
goto err;
len = sizeof(addr);
*fd = accept(listenFD, (struct sockaddr *)&addr, &len);
if (*fd < 0) {
syserror("accept failed");
goto err;
}
debug("accepted connection");
if (!set_socket_buffer_size(*fd)) {
close(*fd);
goto err;
}
stat = 1;
err:
close(listenFD);
return stat;
}
/*
* Set both the send and receive socket buffer sizes.
*/
static int
set_socket_buffer_size(int fd)
{
int size = Req.sock_buf_size;
if (!size)
return 1;
if (setsockopt(fd, SOL_SOCKET, SO_SNDBUF, &size, sizeof(size)) < 0)
return syserror("failed to set send buffer size on socket");
if (setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof(size)) < 0)
return syserror("failed to set receive buffer size on socket");
return 1;
}
static void
datagram_client_bw(int domain)
{
char *buf;
int sockFD;
struct sockaddr_in serverAddr;
datagram_client_init(&sockFD, domain, &serverAddr);
buf = qmalloc(Req.msg_size);
if (!synchronize())
goto err;
while (!Finished) {
int n = sendto(sockFD, buf, Req.msg_size, 0,
(struct sockaddr *)&serverAddr, sizeof(serverAddr));
if (Finished)
break;
if (n < 0) {
LStat.s.no_errs++;
continue;
} else {
LStat.s.no_bytes += n;
LStat.s.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
show_results(BANDWIDTH_SR);
}
static void
datagram_server_bw(int domain)
{
int sockFD;
char *buf = 0;
if (!datagram_server_init(&sockFD, domain))
return;
if (!synchronize())
goto err;
buf = qmalloc(Req.msg_size);
while (!Finished) {
int n = recv(sockFD, buf, Req.msg_size, 0);
if (Finished)
break;
if (n < 0) {
LStat.r.no_errs++;
continue;
} else {
LStat.r.no_bytes += n;
LStat.r.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
}
static void
datagram_client_lat(int domain)
{
char *buf;
int sockFD;
struct sockaddr_in addr;
datagram_client_init(&sockFD, domain, &addr);
buf = qmalloc(Req.msg_size);
if (!synchronize())
goto err;
while (!Finished) {
int n = sendto(sockFD, buf, Req.msg_size, 0,
(struct sockaddr *)&addr, sizeof(addr));
if (Finished)
break;
if (n < 0) {
LStat.s.no_errs++;
continue;
} else {
LStat.s.no_bytes += n;
LStat.s.no_msgs++;
}
n = recv(sockFD, buf, Req.msg_size, 0);
if (Finished)
break;
if (n < 0) {
LStat.r.no_errs++;
continue;
} else {
LStat.r.no_bytes += n;
LStat.r.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
show_results(LATENCY);
}
/*
* Set default IP parameters and ensure that any that are set are being used.
*/
static void
ip_parameters(long msgSize)
{
setp_u32(0, L_MSG_SIZE, msgSize);
setp_u32(0, R_MSG_SIZE, msgSize);
par_use(L_PORT);
par_use(R_PORT);
par_use(L_SOCK_BUF_SIZE);
par_use(R_SOCK_BUF_SIZE);
opt_check();
}
static void
datagram_server_lat(int domain)
{
int sockFD;
char *buf = 0;
if (!datagram_server_init(&sockFD, domain))
goto err;
if (!synchronize())
goto err;
buf = qmalloc(Req.msg_size);
while (!Finished) {
struct sockaddr_in clientAddr;
socklen_t clientLen = sizeof(clientAddr);
int n = recvfrom(sockFD, buf, Req.msg_size, 0,
(struct sockaddr *)&clientAddr, &clientLen);
if (Finished)
break;
if (n < 0) {
LStat.r.no_errs++;
continue;
} else {
LStat.r.no_bytes += n;
LStat.r.no_msgs++;
}
n = sendto(sockFD, buf, Req.msg_size, 0,
(struct sockaddr *)&clientAddr, clientLen);
if (Finished)
break;
if (n < 0) {
LStat.s.no_errs++;
continue;
} else {
LStat.s.no_bytes += n;
LStat.s.no_msgs++;
}
}
Successful = 1;
err:
stop_timing();
exchange_results();
free(buf);
close(sockFD);
}
/*
* Datagram client initialization.
*/
static void
datagram_client_init(int *fd, int domain, struct sockaddr_in *serverAddr)
{
uint32_t port;
struct hostent *host;
struct sockaddr_in clientAddr;
socklen_t clientLen = sizeof(clientAddr);
client_send_request();
*fd = socket(domain, SOCK_DGRAM, 0);
if (*fd < 0)
syserror_die("socket failed");
clientAddr.sin_family = AF_INET;
clientAddr.sin_addr.s_addr = htonl(INADDR_ANY);
clientAddr.sin_port = htons(0);
if (bind(*fd, (struct sockaddr *)&clientAddr, clientLen) < 0)
syserror_die("bind failed");
if (getsockname(*fd, (struct sockaddr *)&clientAddr, &clientLen) < 0)
syserror_die("getsockname failed");
if (!set_socket_buffer_size(*fd))
die();
host = gethostbyname(ServerName);
if (!host)
error_die("cannot find machine %s", ServerName);
serverAddr->sin_family = AF_INET;
if (host->h_length > sizeof(serverAddr->sin_addr))
error_die("address too large to handle");
memcpy(&serverAddr->sin_addr.s_addr, host->h_addr, host->h_length);
if (!recv_mesg(&port, sizeof(port), "port"))
die();
port = decode_port(&port);
debug("sending from %s port %d to %d",
domain == AF_INET ? "UDP" : "RDS", ntohs(clientAddr.sin_port), port);
serverAddr->sin_port = htons(port);
}
/*
* Datagram server initialization.
*/
static int
datagram_server_init(int *fd, int domain)
{
uint32_t port;
struct sockaddr_in addr;
socklen_t len = sizeof(addr);
*fd = socket(domain, SOCK_DGRAM, 0);
if (*fd < 0)
return syserror("socket failed");
memset(&addr, 0, len);
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = htonl(INADDR_ANY);
addr.sin_port = htons(Req.port);
if (bind(*fd, (struct sockaddr *)&addr, len) < 0) {
syserror("bind failed");
goto err;
}
if (getsockname(*fd, (struct sockaddr *)&addr, &len) < 0) {
syserror("getsockname failed");
goto err;
}
if (!set_socket_buffer_size(*fd))
goto err;
encode_port(&port, ntohs(addr.sin_port));
if (!send_mesg(&port, sizeof(port), "port"))
goto err;
return 1;
err:
close(*fd);
return 0;
}
/*
* Send a complete message to a socket. A zero byte write indicates an end of
* file which suggests that we are finished.
*/
static int
send_full(int fd, void *ptr, int len)
{
int n = len;
while (!Finished && n) {
int i = write(fd, ptr, n);
if (i < 0)
return i;
ptr += i;
n -= i;
if (i == 0)
set_finished();
}
return len-n;
}
/*
* Receive a complete message from a socket. A zero byte read indicates an end
* of file which suggests that we are finished.
*/
static int
recv_full(int fd, void *ptr, int len)
{
int n = len;
while (!Finished && n) {
int i = read(fd, ptr, n);
if (i < 0)
return i;
ptr += i;
n -= i;
if (i == 0)
set_finished();
}
return len-n;
}
/*
* Encode a port which is stored as a 32 bit unsigned.
*/
static void
encode_port(uint32_t *p, uint32_t port)
{
enc_init(p);
enc_int(port, sizeof(port));
}
/*
* Decode a port which is stored as a 32 bit unsigned.
*/
static uint32_t
decode_port(uint32_t *p)
{
dec_init(p);
return dec_int(sizeof(uint32_t));
}

113
mkhelp Executable file
View File

@ -0,0 +1,113 @@
#!/usr/bin/env perl
#
use strict;
use warnings;
use diagnostics;
my $help_txt = "help.txt";
my $help_c = "help.c";
my $top = "
/*
* This was generated from $help_txt. Do not modify directly.
*
* Copyright (c) 2002-2007 Johann George. All rights reserved.
* Copyright (c) 2006-2007 QLogic Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
* General Public License (GPL) Version 2, available from the file
* COPYING in the main directory of this source tree, or the
* OpenIB.org BSD license below:
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above
* copyright notice, this list of conditions and the following
* disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials
* provided with the distribution.
*
* THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
char *Usage[] ={
";
my $end = "
0,
};
";
sub panic {
print STDERR @_, "\n";
exit 1;
}
sub main() {
my %defs;
$defs{$_} = 1 for (@ARGV);
my $iFile;
open($iFile, "<", $help_txt) or
panic("cannot find $help_txt");
my $str = "";
my $keep = 1;
while (<$iFile>) {
chomp;
s/\s+$//;
if (/^ /) {
if ($keep) {
s///;
s/(["\\])/\\$1/g;
s/$/\\n/;
if (/^(.{68}(?>[^\\]?))(..*)/) {
$str .= " "x8 . "\"$1\"\n";
$str .= " "x12 . "\"$2\"\n";
} else {
$str .= " "x8 . "\"$_\"\n";
}
}
} else {
my @args = split;
my $arg0 = lc(shift @args);
$keep = 1;
for (@args) {
if (/^\+(.*)/) {
$keep = 0 unless ($defs{$1});
} elsif (/^-(.*)/) {
$keep = 0 if ($defs{$1});
}
}
if ($keep) {
if ($str) {
chop $str;
$str .= ",\n";
}
$str .= " "x4 . "\"$arg0\",\n";
}
}
}
close $iFile;
if ($str) {
chop $str;
$str .= ",\n";
}
$top =~ s/^\n//;
$end =~ s/^\n//;
my $oFile;
open($oFile, ">", $help_c) or
panic("cannot create $help_c");
print $oFile $top, $str, $end;
close $oFile;
}
main();

2913
qperf.c Normal file
View File

File diff suppressed because it is too large Load Diff

316
qperf.h Normal file
View File

@ -0,0 +1,316 @@
/*
* qperf - general header file.
*
* Copyright (c) 2002-2007 Johann George. All rights reserved.
* Copyright (c) 2006-2007 QLogic Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
* General Public License (GPL) Version 2, available from the file
* COPYING in the main directory of this source tree, or the
* OpenIB.org BSD license below:
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above
* copyright notice, this list of conditions and the following
* disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials
* provided with the distribution.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
/*
* Parameters.
*/
#define STRSIZE 64
/*
* For convenience and readability.
*/
#define SUCCESS0 0
#define cardof(a) (sizeof(a)/sizeof(*a))
#define endof(a) (&a[cardof(a)])
#define streq(a, b) (strcmp(a, b) == 0)
#define is_client() (ServerName)
typedef uint64_t CLOCK;
/*
* Time indices.
*/
typedef enum {
T_REAL,
T_USER,
T_NICE,
T_KERNEL,
T_IDLE,
T_IOWAIT,
T_IRQ,
T_SOFTIRQ,
T_STEAL,
T_N
} TIME_INDEX;
/*
* Parameter indices. P_NULL must be 0.
*/
typedef enum {
P_NULL,
L_ACCESS_RECV,
R_ACCESS_RECV,
L_AFFINITY,
R_AFFINITY,
L_FLIP,
R_FLIP,
L_ID,
R_ID,
L_MSG_SIZE,
R_MSG_SIZE,
L_MTU_SIZE,
R_MTU_SIZE,
L_NO_MSGS,
R_NO_MSGS,
L_POLL_MODE,
R_POLL_MODE,
L_PORT,
R_PORT,
L_RATE,
R_RATE,
L_RD_ATOMIC,
R_RD_ATOMIC,
L_SOCK_BUF_SIZE,
R_SOCK_BUF_SIZE,
L_TIME,
R_TIME,
L_TIMEOUT,
R_TIMEOUT,
P_N
} PAR_INDEX;
/*
* What we are measuring.
*/
typedef enum {
LATENCY,
MSG_RATE,
BANDWIDTH,
BANDWIDTH_SR
} MEASURE;
/*
* Request to the server. Note that most of these must be of type uint32_t
* because of the way options are set. The minor version must be changed if
* there is change to this data structure.
*/
typedef struct REQ {
uint16_t ver_maj; /* Major version */
uint16_t ver_min; /* Minor version */
uint16_t ver_inc; /* Incremental version */
uint16_t req_index; /* Request index (into Tests) */
uint32_t flip; /* Flip local/remote node functions */
uint32_t access_recv; /* Access data after receiving */
uint32_t affinity; /* Processor affinity */
uint32_t poll_mode; /* Poll mode */
uint32_t port; /* Port number requested */
uint32_t rd_atomic; /* Number of pending RDMA or atomics */
uint32_t timeout; /* Timeout for messages */
uint32_t msg_size; /* Message Size */
uint32_t mtu_size; /* MTU Size */
uint32_t no_msgs; /* Number of messages */
uint32_t sock_buf_size; /* Socket buffer size */
uint32_t time; /* Duration in seconds */
char id[STRSIZE]; /* Identifier */
char rate[STRSIZE]; /* Rate */
} REQ;
/*
* Transfer statistics.
*/
typedef struct USTAT {
uint64_t no_bytes; /* Number of bytes transfered */
uint64_t no_msgs; /* Number of messages */
uint64_t no_errs; /* Number of errors */
} USTAT;
/*
* Statistics.
*/
typedef struct STAT {
uint32_t no_cpus; /* Number of processors */
uint32_t no_ticks; /* Ticks per second */
uint32_t max_cqes; /* Maximum CQ entries */
CLOCK time_s[T_N]; /* Start times */
CLOCK time_e[T_N]; /* End times */
USTAT s; /* Send statistics */
USTAT r; /* Receive statistics */
USTAT rem_s; /* Remote send statistics */
USTAT rem_r; /* Remote receive statistics */
} STAT;
/*
* Results per node.
*/
typedef struct RESN {
double time_real; /* Real (elapsed) time in seconds */
double time_cpu; /* Cpu time in seconds */
double cpu_total; /* Cpu time (as a fraction of a cpu) */
double cpu_user; /* User time (fraction of cpu) */
double cpu_intr; /* Interrupt time (fraction of cpu) */
double cpu_idle; /* Idle time (fraction of cpu) */
double cpu_kernel; /* Kernel time (fraction of cpu) */
double cpu_io_wait; /* IO wait time (fraction of cpu) */
} RESN;
/*
* Results.
*/
typedef struct RES {
RESN l; /* Local information */
RESN r; /* Remote information */
double send_bw; /* Send bandwidth */
double recv_bw; /* Receive bandwidth */
double msg_rate; /* Messaging rate */
double send_cost; /* Send cost */
double recv_cost; /* Receive cost */
double latency; /* Latency */
} RES;
/*
* Functions prototypes.
*/
void client_send_request(void);
void debug(char *fmt, ...);
void dec_init(void *p);
int64_t dec_int(int n);
void dec_str(char *s, int n);
void die(void);
void enc_init(void *p);
void enc_int(int64_t l, int n);
void enc_str(char *s, int n);
int error(char *fmt, ...);
void error_die(char *fmt, ...);
void exchange_results(void);
int left_to_send(long *sentp, int room);
void opt_check(void);
void *qmalloc(long n);
int recv_mesg(void *ptr, int len, char *item);
int send_mesg(void *ptr, int len, char *item);
void set_finished(void);
void setp_u32(char *name, PAR_INDEX index, uint32_t l);
void setp_str(char *name, PAR_INDEX index, char *s);
void setv_u32(PAR_INDEX index, uint32_t l);
void show_results(MEASURE measure);
void stop_timing(void);
int synchronize(void);
int syserror(char *fmt, ...);
void syserror_die(char *fmt, ...);
void touch_data(void *p, int n);
void par_use(PAR_INDEX index);
/*
* Socket tests (ip.c).
*/
void run_client_rds_bw(void);
void run_server_rds_bw(void);
void run_client_rds_lat(void);
void run_server_rds_lat(void);
void run_client_sdp_bw(void);
void run_server_sdp_bw(void);
void run_client_sdp_lat(void);
void run_server_sdp_lat(void);
void run_client_tcp_bw(void);
void run_server_tcp_bw(void);
void run_client_tcp_lat(void);
void run_server_tcp_lat(void);
void run_client_udp_bw(void);
void run_server_udp_bw(void);
void run_client_udp_lat(void);
void run_server_udp_lat(void);
/*
* InfiniBand tests (ib.c).
*/
void run_client_bug(void);
void run_server_bug(void);
void run_client_rc_bi_bw(void);
void run_server_rc_bi_bw(void);
void run_client_rc_bw(void);
void run_server_rc_bw(void);
void run_client_rc_compare_swap_mr(void);
void run_server_rc_compare_swap_mr(void);
void run_client_rc_fetch_add_mr(void);
void run_server_rc_fetch_add_mr(void);
void run_client_rc_lat(void);
void run_server_rc_lat(void);
void run_client_rc_rdma_read_bw(void);
void run_server_rc_rdma_read_bw(void);
void run_client_rc_rdma_read_lat(void);
void run_server_rc_rdma_read_lat(void);
void run_client_rc_rdma_write_bw(void);
void run_server_rc_rdma_write_bw(void);
void run_client_rc_rdma_write_lat(void);
void run_server_rc_rdma_write_lat(void);
void run_client_rc_rdma_write_poll_lat(void);
void run_server_rc_rdma_write_poll_lat(void);
void run_client_uc_bi_bw(void);
void run_server_uc_bi_bw(void);
void run_client_uc_bw(void);
void run_server_uc_bw(void);
void run_client_uc_lat(void);
void run_server_uc_lat(void);
void run_client_uc_rdma_write_bw(void);
void run_server_uc_rdma_write_bw(void);
void run_client_uc_rdma_write_lat(void);
void run_server_uc_rdma_write_lat(void);
void run_client_uc_rdma_write_poll_lat(void);
void run_server_uc_rdma_write_poll_lat(void);
void run_client_ud_bi_bw(void);
void run_server_ud_bi_bw(void);
void run_client_ud_bw(void);
void run_server_ud_bw(void);
void run_client_ud_lat(void);
void run_server_ud_lat(void);
void run_client_ver_rc_compare_swap(void);
void run_server_ver_rc_compare_swap(void);
void run_client_ver_rc_fetch_add(void);
void run_server_ver_rc_fetch_add(void);
/*
* Variables.
*/
extern RES Res;
extern REQ Req;
extern STAT LStat;
extern char *Usage[];
extern char *TestName;
extern char *ServerName;
extern int Successful;
extern volatile int Finished;