BWCTL Version 1.1a (beta)

(Bandwidth Control)
http://e2epi.internet2.edu/bwctl/

$Date: 2004/09/14 22:55:52 $

BWCTL is a command line client application and a scheduling and policy daemon that wraps Iperf.

From the Iperf web site:
Iperf is a tool to measure maximum TCP bandwidth, allowing the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss.

Currently BWCTL wraps Iperf by actually executing the Iperf command line program on the system. The bwctl client application works by contacting a bwctld process on the two test endpoint systems. BWCTL will work as a 3-party application. The client can arrange a test between two servers on two different systems. If the local system is intended to be one of the endpoints of the test a local bwctld is not required, bwctl will detect that there is no local server and execute the required bwctld functionality directly. bwctld manages and schedules the resources of the host on which it runs. A more detailed description of the architecture is available in the architecture documentation.

The bwctl client is used to request the type of Iperf test wanted. Furthermore, it requests when the test is wanted. bwctld either responds with a tentative reservation or a test denied message. Once bwctl is able to get a matching reservation from both bwctld processes (one for each host involved in the test), it confirms the reservation. Then, the bwctld processes run the test and return the results. The results are returned to the client from both sides of the test. Additionally, the bwctld processes share the results from their respective sides of the test with each other.

BWCTL is used to enable non-specific Iperf tests to a host without having to give full user accounts on the given system. Users want the ability to run Iperf to determine the achievable or available bandwidth between a pair of hosts. It is often useful to test to multiple points along a network path to determine the network characteristics along that path. Typically, users who want to do this path decomposition have to contact the network/system administrators that control the hosts along the path directly. The administrator needs to either run half of the test for the user or provide a user account on the host. Also, network paths of interest usually are controlled by multiple administrators. These hurdles have made this kind of testing difficult in practice.

BWCTL was designed to help with this problem. It allows an administrator to configure a given host as an Iperf endpoint that can be shared by multiple users without concern that those users will interfere with each other. Specific policy limits can be applied to specific users, and individual tests are scheduled so they will not interfere with each other. BWCTL allows the administrator to classify incoming connections based upon a user name and AES key (generated by a pass phrase) combination or, alternatively, based upon an IP/netmask. Once the connection is classified, the bwctld can determine the exact type and intensities of Iperf tests that will be allowed. (More details on the policy controls can be found in the bwctld.limits(8) manual page.)

What's New

3-Party communication is now supported. The client does not have to be on one of the endpoint hosts.

Modified authentication options to allow the client to authenticate to each side of the test using different authentication.

A local bwctld is no longer required. bwctl will detect if there is a local bwctld and use it if it is there, and mimic the functionality if it is not.
Port ranges for the bwctld peer connections can be specified to allow bwctld to work in firewall environments.
Split access logging from error logging in bwctld.
Added support for setting the TOS bits in the test packets.

Features

Supports (and requires) versions 1.7.0 or 2.0.b of Iperf.
Full IPv6 support. No options needed. If the target of a test is specified by a DNS hostname, and that name has both an IPv4 and an IPv6 address defined, the bwctl command line application prefers the IPv6 address.
Data from both sides of the test is returned so that sending rate can be compared with receiving rate.
3-Party communication is supported. The client does not have to be on one of the test endpoint hosts.
A local bwctld is not required. bwctl will detect if there is a local bwctld and use it if it is there, and mimic the functionality if it is not.
Port ranges for the bwctld peer connections can be specified to allow bwctld to work in firewall environments.
Port ranges for the tests can be specified to allow passive monitoring to characterize the test streams more easily.
Limits the types of Iperf tests that can be run. Simultaneous connections are not supported. Sending a set amount of data is not supported because BWCTL needs to know how long the test will take before the test is started.
Supports dynamic TCP window size determination. (Currently, this is limited because the bottleneck capacity is specified as a constant in the config file. In the future, bottleneck capacity will most likely be dynamically determined using packet dispersion techniques.)

Requirements

BWCTL currently requires Iperf version 1.7.0 (or 2.0.b for the adventurous).
The bwctld daemon requires a reasonably synchronized clock. It is scheduling tests and must be sure it has the same idea of when a test should take place as does the peer system with which the test is being done. Therefore, BWCTL requires that NTP be running to synchronize the system clock. This means that NTP needs to be setup correctly on the system so that NTP can actually calculate a reasonable estimate of the time error. For the NTP algorithms to work correctly, NTP MUST be configured with no fewer than 4 clocks. (See http://twiki.ntp.org/bin/view/Support/SelectingOffsiteNTPServers for more details.)
Additionally, BWCTL (bwctl and bwctld) uses the NTP-specific system calls to fetch time and time error estimates from the system kernel, so the kernel must support these interfaces. All modern UNIX systems support them.
gnumake may be required for the build process (see Building the Application).
To get good results, the end hosts will almost certainly need to be well tuned. See http://www.psc.edu/networking/perf_tune.html and http://www-didc.lbl.gov/TCP-tuning/buffers.html for some excellent tips in that regard.

Supported Systems

BWCTL has been tested most extensively on FreeBSD 4.7 and Linux 2.4.20 but most recent versions of UNIX should work.

Recommended Hardware

BWCTL does not have any strict hardware requirements. The limits imposed by the specific Iperf tests are more taxing to hardware than the scheduling and policy functions added by BWCTL. The hardware requirements are therefore a function of the specific Iperf tests that are desired. Internet2 has had good luck using the following hardware to collect data on the Abilene network:

Intel SCB2 motherboard

2 x 1.266 GHz PIII, 512 KB L2 cache, 133 MHz FSB

2 x 512 MB ECC registered RAM (one/slot to enable interleaving)

2 x Seagate 18 GB SCSI (ST318406LC)

SysConnect Gigabit Ethernet SK-9843 SX

Building the Application

The BWCTL software uses the gnu autoconf tools to configure the build process. BWCTL is distributed with a pre-built configure script so the actual autoconf tools should not be needed on the target system. (Although, gnumake may be required...) The configure script can be run with the --help option to determine the full set of configurable options.

A basic build procedure would be:

% gzip -cd bwctl-$VERS.tar.gz | tar xf -
% cd bwctl-$VERS
  # --prefix is only needed if you don't like the default
  #   (/usr/local on most systems)
% ./configure --prefix=/inst/root
% make
% make install

Please report any build problems to bwctl-users@internet2.edu.

Configuring bwctld

The basic procedure to configure bwctld is to create a bwctld.conf and, optionally, a bwctld.limits file and a bwctld.keys file. These files need to be installed in the same directory that is specified with the -c option to bwctld.. The recommended directory is /inst/root/etc. (The etc directory below your install root.) There are examples of these files in the bwctl-$VERS/conf sub directory of the distribution.

bwctld.conf:
Used to configure basic operation of the daemon such as server listening port, the path for Iperf, and error logging. For a detailed description of the options available, see the bwctld.conf(8) manual page.

bwctld.limits:
Used to configure the policy limits for the daemon. There are two parts to this policy: 1) authentication and 2) authorization. The authentication is done by assigning a class to each new connection. Each class has a set of limits associated with it. The classes are hierarchical, so a connection must pass the limit restrictions of the assigned class as well as all parent classes. For a detailed description of the options available, see the bwctld.limits(8) manual page.

bwctl.keys:
Used to associate identities with AES keys. These identities are then mapped to a class by the bwctld.limits file. For a more detailed description, see the bwctld.keys(8) manual page.

Running bwctld

The normal command-line to start the daemon is:

 % bwctld -c /inst/root/etc

It is possible to run the daemon without a config file directory if enough command-line options are specified, but it is easier to use a config file.

To see all the available options:

 % bwctld -h

More details on running the daemon, as well as a complete description of the command-line options, is available from the bwctld manual page.

Running bwctl

The basic command line for the client is:

% bwctl [options] [-c catchhost] [-s sendhost]

The -c option indicates that the catchhost should be an Iperf server (think packet catcher). The -s option indicates the sendhost should be an Iperf client (think packet sender). At least one of the -c/-s options must be specified. If only one of these options is specified, the local host is assumed to be the other test end point.

To see a list of available options:

% bwctl -h

More details on running the client application, with a complete description of all command-line options, is available from the bwctl manual page.

Mailing Lists

There are two email lists to support this software:

bwctl-users: A general discussion list for users to discuss problems. I will monitor this list as I expect bug reports to be sent here.
bwctl-announce: This list will be used to announce new versions or significant developments with regard to the software.

Information about these lists, including links to subscribe, is at https://mail.internet2.edu/wws/lists/engineering.

Author

Jeff Boote
boote@internet2.edu
Internet2


$Id: index.html,v 1.16 2004/09/14 22:55:52 boote Exp $