Tech Support Notes

Bandwith and Speed testing utilities

nuttcp

nuttcp is a network performance measurement tool intended for use by network and system managers. Its most basic usage is to determine the raw TCP (or UDP) network layer throughput by transferring memory buffers from a source system across an interconnecting network to a destination system

Common Flags for nuttcp

-i1 to watch tests run (1 second intervals)
-w8m to set socket buffers ("window") to 8 MBytes
-u for UDP tests
-R10m for a 10 Mbps UDP test (or TCP rate limit)
-l512 to set UDP packet length (or TCP write size)
-T# - # of times

On the server you want to test start nuttcp

nuttcp -S ( on server )

On the client server you would run the following

nuttcp -T20 -i1 hostname/IP

If you want to set the Window Size you can use the -w flag:

nuttcp -i1 -w8m hostname/IP

Send 300 Mbps of UDP in bursts of 50 packets for 5 seconds

[hyperion] (~) >>> nuttcp -u -Ri300m/50 -i 1 -T5 callisto.example.com
    0.0352 MB /   1.00 sec =    0.2949 Mbps     0 /    36 ~drop/pkt  0.00 ~%loss
    0.0361 MB /   1.00 sec =    0.3031 Mbps     0 /    37 ~drop/pkt  0.00 ~%loss
    0.0352 MB /   1.00 sec =    0.2949 Mbps     0 /    36 ~drop/pkt  0.00 ~%loss
    0.0361 MB /   1.00 sec =    0.3031 Mbps     0 /    37 ~drop/pkt  0.00 ~%loss
    0.0361 MB /   1.00 sec =    0.3031 Mbps     0 /    37 ~drop/pkt  0.00 ~%loss

    0.1787 MB /   5.00 sec =    0.2998 Mbps 99 %TX 0 %RX 0 / 183 drop/pkt 0.00 %loss

iperf

You need to install iperf on both systems. If you are running Cent/Rhel 5 then a simple yum install iperf will do the trick. If you are on Cent/Rhel 6 then you need to enable the epel repository using this guide. If you are running Debian/Ubuntu just run apt-get install iperf and you're good to go. For Windows you can grab the installer from iperf.fr

Linux instructions

To use the utility you need to launch it on both servers that you are testing for throughput issues. On the client side you invoke the command with the -c flag. On the server side you use -s. Here is a basic test:

There are a lot of things we can do to make this output better with more meaningful data. For example, let's say we want the test to run for 20 seconds instead of 10 (-t 20), and we want to display transfer data every 2 seconds instead of the default of 10 (-i 2), and we want to test on port 8000 instead of 5001 (-p 8000). For the purposes of this exercise, let's use those customization as our baseline. This is what the command string would look like on both ends:

Server Side

[saturn] (~) >>> iperf -s -i 2

Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)

[  4] local 192.168.1.81 port 5001 connected with 192.168.1.83 port 52458
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0- 2.0 sec  1.57 MBytes  6.60 Mbits/sec
[  4]  2.0- 4.0 sec   641 KBytes  2.63 Mbits/sec
[  4]  4.0- 6.0 sec   675 KBytes  2.76 Mbits/sec
[  4]  6.0- 8.0 sec  1.66 MBytes  6.95 Mbits/sec
[  4]  8.0-10.0 sec  1.99 MBytes  8.34 Mbits/sec
[  4] 10.0-12.0 sec  2.32 MBytes  9.72 Mbits/sec
[  4] 12.0-14.0 sec  2.84 MBytes  11.9 Mbits/sec
[  4] 14.0-16.0 sec  1.46 MBytes  6.12 Mbits/sec
[  4] 16.0-18.0 sec  1.17 MBytes  4.89 Mbits/sec
[  4] 18.0-20.0 sec  1.26 MBytes  5.27 Mbits/sec
[  4]  0.0-20.6 sec  16.0 MBytes  6.53 Mbits/sec

Client Side

root@river:~# iperf -c 192.168.1.81 -t 20 -i 2

Client connecting to 192.168.1.81, TCP port 5001
TCP window size: 23.5 KByte (default)

[  3] local 192.168.1.83 port 52458 connected with 192.168.1.81 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 2.0 sec  1.75 MBytes  7.34 Mbits/sec
[  3]  2.0- 4.0 sec   640 KBytes  2.62 Mbits/sec
[  3]  4.0- 6.0 sec   640 KBytes  2.62 Mbits/sec
[  3]  6.0- 8.0 sec  1.88 MBytes  7.86 Mbits/sec
[  3]  8.0-10.0 sec  1.88 MBytes  7.86 Mbits/sec
[  3] 10.0-12.0 sec  2.38 MBytes  9.96 Mbits/sec
[  3] 12.0-14.0 sec  2.88 MBytes  12.1 Mbits/sec
[  3] 14.0-16.0 sec  1.38 MBytes  5.77 Mbits/sec
[  3] 16.0-18.0 sec  1.25 MBytes  5.24 Mbits/sec
[  3] 18.0-20.0 sec  1.25 MBytes  5.24 Mbits/sec
[  3]  0.0-20.3 sec  16.0 MBytes  6.60 Mbits/sec
root@river:~#

For some additional guides for using iperf see the following sites:
Using iperf to troubleshoot speed/throughput issues
Iperf: A simple but powerful tool for troubleshooting networks
Iperf: The easy tutorial

Speedtest CLI

Command line interface for testing internet bandwidth using speedtest.net

Speedtest cli

[root@toronto ~]# speedtest-cli --list > list
[root@toronto ~]# grep USA list
3824) Hot Spot Broadband (Reno, NV, USA) [3367.02 km]
2201) LAUSA (Gloucester, Great Britain) [5540.26 km]
5629) PROVEDOR INTERSOUSA (CAFARNAUM, Brazil) [7265.39 km]

[root@toronto ~]# speedtest-cli --server 3824
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from SoftLayer Technologies (158.85.74.167)...
Hosted by Hot Spot Broadband (Reno, NV) [3367.02 km]: 97.847 ms
Testing download speed........................................
Download: 101.66 Mbit/s
Testing upload speed..................................................
Upload: 23.39 Mbit/s
[root@toronto ~]#

[root@toronto ~]# grep Toronto list
4049) VMedia Inc (Toronto, ON, Canada) [28.04 km]
3860) Start Communications (Toronto, ON, Canada) [28.04 km]
4441) Keewaytinook Okimakanak (Toronto, ON, Canada) [28.04 km]
5979) SoftLayer Technologies, Inc. (Toronto, ON, Canada) [28.04 km]
3025) Beanfield Metroconnect (Toronto, ON, Canada) [28.04 km]
3575) TELUS (Toronto, ON, Canada) [28.04 km]
2565) TekSavvy Solutions (Toronto, ON, Canada) [28.04 km]
 826) Nexicom Inc. (Toronto, ON, Canada) [28.04 km]
5210) ZEROFAIL (Toronto, ON, Canada) [28.04 km]

[root@toronto ~]# speedtest-cli --server 4049
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from SoftLayer Technologies (158.85.74.167)...
Hosted by VMedia Inc (Toronto, ON) [28.04 km]: 21.135 ms
Testing download speed........................................
Download: 639.78 Mbit/s
Testing upload speed..................................................
Upload: 60.59 Mbit/s
[root@toronto ~]#

[root@toronto ~]# grep Houston list
4043) SoftLayer Technologies, Inc. (Houston, TX, United States) [2117.86 km]
1763) Comcast (Houston, TX, United States) [2117.86 km]

[root@toronto ~]# speedtest-cli --server 4043
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from SoftLayer Technologies (158.85.74.167)...
Hosted by SoftLayer Technologies, Inc. (Houston, TX) [2117.86 km]: 872.729 ms
Testing download speed........................................
Download: 330.09 Mbit/s
Testing upload speed..................................................
Upload: 49.55 Mbit/s
[root@toronto ~]#

iptraf

more here about the tool and uses
Iptraf tutorial
Iptraf and iftop walkthrough

iftop

more here about the tool and uses
Iptraf and iftop walkthrough
Iftop guide

ethtool

Utility to query or control network driver and hardware settings. To show ethernet card properties such as speed, wake on, duplex and the link detection status just pass the device name

[saturn] (~) >>> ethtool eth0
Settings for eth0:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Speed: 10Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes
Using ethtool you can change the speed of the ethernet device.
[saturn] (~) >>> ethtool -s eth0 speed 100 autoneg off

When you change the speed when the adapter is online, it automatically goes offline. To bring the device back online you need to use the ifup command.

ifup eth0

Other handy flag examples here with usage
-a [Show the Auto-negotiation, RX and TX variables]
-S [Display the bytes transfered, received, errors, etc. Basicaly all the NIC stats]

Note that the -S option is very verbose. Here is a small snippet of the output:

[root@houston ~]# ethtool -S eth1
NIC statistics:
     rx_packets: 171765
     tx_packets: 73174
     rx_bytes: 202950273
     tx_bytes: 9992856
     rx_broadcast: 3513
     tx_broadcast: 2
     rx_multicast: 0
     tx_multicast: 5
     multicast: 0
     collisions: 0

[root@houston ~]# ethtool -S eth1 > output
[root@houston ~]# cat output | wc -l
115

You can also get infromation about the driver using the -i flag

[root@houston ~]# ethtool -i eth0
driver: igb
version: 5.0.5-k
firmware-version: 3.29, 0x8000023b
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

tcpdump

tcpdump is a common packet analyzer that runs under the command line. It allows the user to intercept and display TCP/IP and other packets being transmitted or received over a network to which the computer is attached.
A benefit of tcpdump is the use of expressions. There are three main types of expression: type, dir, and proto.

  1. Type options are host, net, and port
  2. Direction is indicated by dir, and there you can have src, dst, src or dst, and src and dst
  3. Proto - works for tcp, udp, and icmp. Note that you don't have to type proto

Common flags for tcpdump

-v = verbose output
-X = adds payload information
-s = increase snaplength

Here's a capture of exactly two (-c2) ICMP packets (a ping and pong). On host #1 with private IP 10.8.x.x we start a ping to host 10.5.x.x

[callisto] (~) >>> ping -c20 10.5.x.x
On host #2. we capture 2 icmp requests

[hyperion] (~) >>>  tcpdump -nS -c2 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
12:00:59.475571 IP 10.8.x.x > 10.5.x.x: ICMP echo request, id 884, seq 1, length 64
12:00:59.475617 IP 10.5.x.x > 10.8.x.x: ICMP echo reply, id 884, seq 1, length 64
2 packets captured
2 packets received by filter
0 packets dropped by kernel
For more verbose information you can use the following command tcpdump -nnvXSs 0 -c2 icmp