Internet2

Bandwidth Test Controller (BWCTL)

About    |    Downloads      |     Manual Pages     |     Cookbook (PDF)    |    License

Of Note

Mailing List:
Two mailing lists (perfsonar-announce and perfsonar-user) are hosted at the domaininternet2.edu; both are publicly archived and open to subscriptions.

Get Involved!

Make a difference. Get involved in one of the many Internet2 community working or special interest groups.

BWCTL BWCTL is a command line client application and a scheduling and policy daemon that wraps the network measurement tools, including Iperf, Iperf3, Nuttcp, Ping, Traceroute, Tracepath, and OWAMP. These tests can measure maximum TCP bandwidth, with various tuning options available, as well as the delay, jitter, and loss rate of a network.

The BWCTL client application works by contacting a BWCTL server process on the two test endpoint systems. BWCTL will work as a 3-party application. The client can arrange a test between two servers on two different systems. If the local system is intended to be one of the endpoints of the test, the BWCTL client will detect whether a local BWCTL server is running and will handle the required server functionality if needed. The BWCTL server manages and schedules the resources of the host on which it runs.

The BWCTL client is used to request the type of measurement test wanted. Furthermore, it requests when the test should be executed. The BWCTL server either responds with a tentative reservation or a test denied message. Once the BWCTL clent is able to get a matching reservation from both BWCTL servers (one for each host involved in the test), it confirms the reservation. Then, the BWCTL servers run the test and return the results. The results are returned to the client from both sides of the test. Additionally, the BWCTL servers share the results from their respective sides of the test with each other.

BWCTL is used to enable non-specific measurement tests to a host without having to give full user accounts on the given system. Users want the ability to run measurement tests to determine the achievable or available bandwidth, path, one-way latency or loss between a pair of hosts. It is often useful to test to multiple points along a network path to determine the network characteristics along that path. Typically, users who want to do this path decomposition have to contact the network/system administrators that control the hosts along the path directly. The administrator needs to either run half of the test for the user or provide a user account on the host. Also, network paths of interest usually are controlled by multiple administrators. These hurdles have made this kind of testing difficult in practice.

BWCTL was designed to help with this problem. It allows an administrator to configure a given host as a measurement endpoint that can be shared by multiple users without concern that those users will interfere with each other. Specific policy limits can be applied to specific users, and individual tests are scheduled so they will not interfere with each other. BWCTL allows the administrator to classify incoming connections based upon a user name and AES key (generated by a pass phrase) combination or, alternatively, based upon an IP/netmask. Once the connection is classified, the BWCTL server can determine the exact type and intensities of measurement tests that will be allowed. (More details on the policy controls can be found in the bwctld.limits(8) manual page.)


Features

  • Support for Iperf, Iperf3 and Nuttcp tests.
  • Support for Ping tests.
  • Support for OWAMP (One-Way Latency) tests.
  • Support for Traceroute and Tracepath tests.
  • Full IPv6 support. No options needed. If the target of a test is specified by a DNS hostname, and that name has both an IPv4 and an IPv6 address defined, the the BWCTL client prefers the IPv6 address.
  • Data from both sides of the test is returned so that sending side measurements can be compared with receiving side.
  • Third party communication is supported. The client does not have to be on one of the test endpoint hosts.
  • A local BWCTL server is not required. the BWCTL client will detect if there is a local BWCTL server and use it if it is there; the BWCTL client will mimic the missing functionality if needed.
  • Port ranges for the BWCTL Server peer connections as well as the measurement tests can be specified to allow the BWCTL Server to work in firewall environments.
  • Limits the types of measurement tests that can be run.


Requirements

  • The BWCTL server requires NTP be used to synchronize the system clock so it can be sure that two endpoints of the test are actually agreeing to the same scheduled time window for test execution. (See the allow_unsync configuration option described in the bwctld.conf(8) manual page to see how to by-pass this requirement.)

    See http://support.ntp.org/bin/view/Support/ConfiguringNTP for more details about configuring NTP

  • To get good performance results, the endpoints will need to be well tuned. See http://fasterdata.es.net/ for some excellent tips in that regard.


Supported Systems

BWCTL has been tested most extensively on Red Hat Linux 5 and 6 systems, but most recent versions of UNIX or OS X should work.


Running The BWCTL Client

The basic command lines for the client are:
% bwctl [options] [-c catchhost] [-s sendhost]
% bwping [options] [-c catchhost] [-s sendhost]
% bwtraceroute [options] [-c catchhost] [-s sendhost]

The bwctl command is used to run throughput tests. The bwping command is used to run latency tests. The bwtraceroute command is used to run traceroute commands.

The -c option indicates that the catchhost should be the receiving host for the measurements (think packet catcher). The -s option indicates the sendhost should be the sending host for the measurements. At least one of the -c/-s options must be specified. If only one of these options is specified, the local host is assumed to be the other test endpoint.

To see a list of available options:

% bwctl -h
% bwping -h
% bwtraceroute -h
More details on running the client application, with a complete description of all command line options, is available from the bwctl manual page.

Sample bwctl commands:

  • bwctl -c receive_host
  • This will use your host as the sender, and run a 10 second iperf test.
  • bwctl -c receive_host -t 30 -f m -i 1
  • This will use your host as the sender, run a 30 second test, format the results in Megabits/sec. Also output results every 2 secs for both the client and server.
  • bwctl -c receive_host:55555 -t 30 -f m -w 16M -i 1
  • This will run the same test, but will connect to the bwctl daemon on the remote host that is running on the non-standard port 55555 (more on ports below). The TCP window will be 16MB.
  • bwctl -c receive_host -s send_host -t 30 -f m
  • This is very useful if you are not logged into one of the two endpoints. It runs a 30 second test from send_host to recv_host.
  • bwctl -L 1000 -c receive_host -t 30 -f m -i 1
  • By default bwctl exits if it can not get a test slot within 5 minutes. Use the -L flag for heavily used servers where you might have to wait longer.
  • bwctl -T iperf3 -c receive_host -s send_host -i 1 -u -b 500M
  • Use iperf3 instead of iperf, and do a 500Mbps UDP test.

Sample bwping commands:

  • bwping -s send_host -c receive_host
  • run a ping from host A to host B
  • bwping -T owamp -s send_host-c receive_host -N 1000 -i .01
  • run owping from host A to host B, sending 1000 packets, spaced .01 seconds apart.

Sample bwtraceroute commands:

  • bwtraceroute -c receive_host -s send_host
  • run a tracetroute from host A to host B
  • bwtraceroute -T tracepath -c receive_host -l 8192 -s send_host
  • run a tracepath from host A to host B, using a packet size of 8192 bytes. This will help find MTU issues.


Configuring The BWCTL server

The basic procedure to configure bwctld is to create a bwctld.conf and, optionally, a bwctld.limits file and a bwctld.keys file. These files need to be installed in the same directory that is specified with the -c option to bwctld. The recommended directory is /install/root/etc. (The etc directory below your install root.) There are examples of these files in the bwctl-$VERS/conf  sub directory of the distribution.

bwctld.conf:
Used to configure basic operation of the daemon, such as server listening port, the path for Iperf, and error logging. For a detailed description of the options available, see the bwctld.conf(8) manual page.

bwctld.limits:
Used to configure the policy limits for the daemon. There are two parts to this policy: 1) authentication and 2) authorization. The authentication is done by assigning a class to each new connection. Each class has a set of limits associated with it. The classes are hierarchical, so a connection must pass the limit restrictions of the assigned class as well as all parent classes. For a detailed description of the options available, see the bwctld.limits(8) manual page.

bwctl.keys:
Used to associate identities with AES keys. These identities are then mapped to a class by the bwctld.limits file. For a more detailed description, see the bwctld.keys(8) manual page.


Running The BWCTL Server

The normal command line to start the daemon is:
 % bwctld -c /install/root/etc
It is possible to run the daemon without a config file directory if enough command line options are specified, but it is easier to use a config file.

To see all the available options:

 % bwctld -h
More details on running the daemon, as well as a complete description of the command line options, is available from the bwctld manual page.


Configuring firewalls

If a firewall is enabled on the machine running bwctld, it will need to be configured to allow incoming clients and tests.

BWCTL uses three different sets of ports:

  • Main daemon listening port for control connection (Default: TCP/4823)
  • bwctld peer connections (Default: TCP/ephemeral range)
    Defined using the peer_port configuration option from bwctld.conf
  • Test Connections (Default: 5000-5900) Defined using the test_port configuration option from bwctld.conf. Each tester can have its own port defined in bwctld.conf. The configuration option for each tester takes the form testername_port. For example, to set the Iperf3 port, you would use the option iperf3_port.


Building the Application

The BWCTL software uses the GNU Autoconf tools to configure the build process. BWCTL is distributed with a pre-built configure script so the actual autoconf tools should not be needed on the target system. (Although, GNU Make is required.) The configure script can be run with the --help option to determine the full set of configurable options.

A basic build procedure would be:

% tar xzf bwctl-$VERS.tar.gz
% cd bwctl-$VERS
  # --prefix is only needed if you don't like the default
  #   (/usr/local on most systems)
% ./configure --prefix=/install/root
% make
% make install
Please report any build problems to perfsonar-users@internet2.edu.


Mailing Lists

There are two email lists to support this software:

  • perfsonar-users: This is a general discussion list for users to discuss problems. This list is monitored as bug reports should be sent here.
  • perfsonar-announce: This list is used to announce new versions or significant developments with regard to the software.


Maintainers

Aaron Brown (aaron@internet2.edu)


Acknowledgments

The following individuals have greatly aided in the development, testing, or debugging of BWCTL:
  • Jeff Boote (Sandia) - Initial Developer
  • Shumon Huque (UPenn)
  • Paul Hyder (NOAA)
  • Warren Matthews (GaTech)
  • Sean Peisert (SDSC)
  • Chris Tracy (MAX)