resperf

Langue: en

Autres versions - même langue

Version: December 4, 2007 (fedora - 04/07/09)

Section: 1 (Commandes utilisateur)

NAME

resperf - test the resolution performance of a caching DNS server

SYNOPSIS

resperf [ -d datafile ] [ -s server_addr ] [ -p port ] [ -b bufsize ] [ -f family ] [ -e ] [ -D ] [ -y name:secret ] [ -A ] [ -h ] [ -i interval ] [ -m max_qps ] [ -P plot_data_file ] [ -r rampup_time ] [ -L max_loss ]

DESCRIPTION

resperf is a companion tool to dnsperf. dnsperf was primarily designed for benchmarking authoritative servers, and is does not work well with caching servers that are talking to the live Internet. One reason for this is that dnsperf uses a "self-pacing" approach, which is based on the assumption that you can keep the server 100% busy simply by sending it a small burst of back-to-back queries to fill up network buffers, and then send a new query whenever you get a response back. This approach works well for authoritative servers that process queries in order and one at a time; it also works pretty well for a caching server in a closed laboratory environment talking to a simulated Internet that's all on the same LAN. Unfortunately, it does not work well with a caching server talking to the actual Internet, which may need to work on thousands of queries in parallel to achieve its maximum throughput. There have been numerous attempts to use dnsperf (or its predecessor, queryperf) for benchmarking live caching servers, usually with poor results. Therefore, a separate tool designed specifically for caching servers is needed.

HOW RESPERF WORKS

Unlike the "self-pacing" approach of dnsperf, resperf works by sending DNS queries at a controlled, steadily increasing rate. By default, resperf will send traffic for 60 seconds, linearly increasing the amount of traffic from zero to 100,000 queries per second.

During the test, resperf listens for responses from the server and keeps track of response rates, failure rates, and latencies. It will also continue listening for responses for an additional 40 seconds after it has stopped sending traffic, so that there is time for the server to respond to the last queries sent. This time period was chosen to be longer than the overall query timeout of both Nominum CNS and current versions of BIND.

If the test is successful, the query rate will at some point exceed the capacity of the server and queries will be dropped, causing the response rate to stop growing or even decrease as the query rate increases.

The result of the test is a set of measurements of the query rate, response rate, failure response rate, and average query latency as functions of time. These are written to a file in a tabular format for plotting using gnuplot or some similar plotting tool. The server's maximum throughput can be determined from the plot either as the highest response rate on the plot, or alternatively as the response rate at the point where a significant number of queries begin to be dropped.

WHAT YOU WILL NEED

Benchmarking a live caching server is serious business. A fast caching server like Nominum CNS running on an Opteron server, resolving a mix of cacheable and non-cacheable queries typical of ISP customer traffic, is capable of resolving more than 50,000 queries per second. In the process, it will send more than 20,000 queries per second to authoritative servers on the Internet, and receive responses to most of them. Assuming an average request size of 50 bytes and a response size of 100 bytes, this amounts to some 8 Mbps of outgoing and 16 Mbps of incoming traffic. If your Internet connection can't handle the bandwidth, you will end up measuring the speed of the connection, not the server, and may saturate the connection causing a degradation in service for other users.

Make sure there is no stateful firewall between the server and the Internet, because most of them can't handle the amount of UDP traffic the test will generate and will end up dropping packets, skewing the test results. Some will even lock up or crash.

You should run resperf on a machine separate from the server under test, on the same LAN. Preferably, this should be a Gigabit Ethernet network. The machine running resperf should be at least as fast as the machine being tested; otherwise, it may end up being the bottleneck.

There should be no other applications running on the machine running resperf. Performance testing at the traffic levels involved is essentially a hard real-time application - consider the fact that at a query rate of 100,000 queries per second, if resperf gets delayed by just 1/100 of a second, 1000 incoming UDP packets will arrive in the meantime, which is more than most operating systems will buffer.

Because the granularity of the timers provided by operating systems is typically too coarse to accurately schedule packet transmissions at sub-millisecond intervals, resperf will busy-wait between packet transmissions, constantly polling for responses. Therefore, it is normal for resperf to consume 100% CPU during the whole test run, even during periods where query rates are relatively low.

You will also need a set of test queries in the dnsperf file format. See the dnsperf man page for instructions on how to construct this query file. To make the test as realistic as possible, the queries should be derived from recorded production client DNS traffic, without removing duplicate queries or other filtering. With the default settings, resperf will use up to 3 million queries in each test run.

If the caching server to be tested has a configurable limit on the number of simultaneous resolutions, like the "max-recursive-clients" statement in Nominum CNS or the "recursive-clients" option in BIND 9, you will probably have to increase it. As a starting point, we recommend a value of 10000 for Nominum CNS and 100000 for BIND 9. Should the limit be reached, it will show up in the plots as an increase in the number of failure responses.

For maximum realism, you could "prime" the cache of the server by having it resolve typical query traffic for some period of time before the test, so that the cache is not empty when the test starts. However, experience has shown that this is not really necessary, as the server will reach a reasonable cache hit rate (70% or more) during the test even when starting with an empty cache. If you do prime the cache, make sure not to use the same set of queries as in the actual test, since that would make the server answer almost all queries from the cache and yield inflated performance numbers.

RUNNING THE TEST

When running resperf, you will need to specify at least the server IP address and the query data file. A typical invocation will look like

           resperf -s 10.0.0.2 -d queryfile
         
 
 

With default settings, the test run will take at most 100 seconds (60 seconds of ramping up traffic and then 40 seconds of waiting for responses), but in practice, the 60-second traffic phase will usually be cut short. To be precise, resperf can transition from the traffic-sending phase to the waiting-for-responses phase in three different ways:

Running for the full allotted time and successfully reaching the maximum query rate (by default, 60 seconds and 100,000 qps, respectively). Since this is a very high query rate, this will rarely happen (with today's hardware); one of the other two conditions listed below will usually occur first.
Exceeding 65,536 outstanding queries. This often happens as a result of (successfully) exceeding the capacity of the server being tested, causing the excess queries to be dropped. The limit of 65,536 queries comes from the number of possible values for the ID field in the DNS packet. Resperf needs to allocate a unique ID for each outstanding query, and is therefore unable to send further queries if the set of possible IDs is exhausted.
When resperf finds itself unable to send queries fast enough. Resperf will notice if it is falling behind in its scheduled query transmissions, and if this backlog reaches 1000 queries, it will print a message like "Fell behind by 1000 queries" (or whatever the actual number is at the time) and stop sending traffic.

Regardless of which of the above conditions caused the traffic-sending phase of the test to end, you should examine the resulting plots to make sure the server's response rate is flattening out toward the end of the test. If it is not, then you are not loading the server enough. If you are getting the "Fell behind" message, make sure that the machine running resperf is fast enough and has no other applications running.

You should also monitor the CPU usage of the server under test. It should reach close to 100% CPU at the point of maximum traffic; if it does not, you most likely have a bottleneck in some other part of your test setup, for example, your external Internet connection.

As resperf runs, some status messages and summary statistics will be written to standard output, and the table of plot data is written to the file resperf.gnuplot in the current directory (or some other file name given with the -P command line option).

THE PLOT DATA FILE

For purposes of generating the plot data file, the test run is divided into time intervals of 0.5 seconds (or some other length of time specified with the -i command line option). Each line in the plot data file corresponds to one such interval, and contains the following values as floating-point numbers:

Time
The midpoint of this time interval, in seconds since the beginning of the run
Target queries per second
The number of queries per second scheduled to be sent in this time interval
Actual queries per second
The number of queries per second actually sent in this time interval
Responses per second
The number of responses received corresponding to queries sent in this time interval, divided by the length of the interval
Failures per second
The number of responses received corresponding to queries sent in this time interval and having an RCODE other than NOERROR or NXDOMAIN, divided by the length of the interval
Average latency
The average time between sending the query and receiving a response, for queries sent in this time interval

Note that the measurements for any given query are always applied to the time interval when the query was sent, not the one when the response (if any) was received. This makes it it easy to compare the query and response rates; for example, if no queries are dropped, the query and response curves will be identical. As another example, if the plot shows 10% failure responses at t=5 seconds, this means that 10% of the queries sent at t=5 seconds eventually failed, not that 10% of the responses received at t=5 seconds were failures.

PLOTTING THE RESULTS

Resperf comes with a shell script, resperf-report, which will run resperf with its output redirected to a file and then automatically generate an illustrated report in HTML format. Command line arguments given to resperf-report will be passed on unchanged to resperf.

You need to have gnuplot installed, because resperf-report uses it to generate the plots. Make sure your version of gnuplot supports the "gif" terminal driver.

The report will be stored with a unique file name based on the current date and time, e.g., 20060812-1550.html. The GIF images of the plots and other auxiliary files will be stored in separate files beginning with the same date-time string. If you need to copy the report to a separate machine for viewing, make sure to copy the .gif files along with the .html file (or simply copy all the files, e.g., using scp 20060812-1550.* host:directory/).

For example, to benchmark a server running on 10.0.0.2, you could run

           resperf-report -s 10.0.0.2 -d queryfile
         
 
 
and then open the resulting .html file in a web browser.

INTERPRETING THE RESULTS

The summary statistics printed on standard output at the end of the test include the server's measured maximum throughput. By default, this is simply the highest point on the response rate plot, without regard to the number of queries being dropped or failing at that point.

You can also make resperf report the throughput at the point in the test where the percentage of queries dropped exceeds a given limit (or the maximum as above if the limit is never exceeded). This can be a more realistic indication of how much the server can be loaded while still providing an acceptable level of service. This is done using the -L command line option; for example, specifying -L 10 makes resperf report the highest throughput reached before the server starts dropping more than 10% of the queries.

There is no corresponding way of automatically constraining results based on the number of failed queries, because unlike dropped queries, resolution failures will occur even when the the server is not overloaded, and the number of such failures is heavily dependent on the query data and network conditions. Therefore, the plots should be manually inspected to ensure that there is not an abnormal number of failures.

OPTIONS

-d datafile
Specifies the input data file. If not specified, resperf will read from standard input.
-s server_addr
Specifies the name or address of the server to which requests will be sent. The default is the loopback address, 127.0.0.1.
-p port
Sets the port on which the DNS packets are sent. If not specified, the standard DNS port (53) is used.
-b bufsize
Sets the size of the socket's send and receive buffers, in kilobytes. If not specified, the default value is 32k.
-f family
Specifies the address family used for sending DNS packets. The possible values are "inet", "inet6", or "any". If "any" (the default value) is specified, resperf will use whichever address family is appropriate for the server it is sending packets to.
-e
Enables EDNS0 [RFC2671], by adding an OPT record to all packets sent.
-D
Sets the DO (DNSSEC OK) bit [RFC3225] in all packets sent. This also enables EDNS0, which is required for DNSSEC.
-y name:secret
Add a TSIG record [RFC2845] to all packets sent, using the specified TSIG key name and secret, where the secret is expressed as a base-64 encoded string.
-A
Reports the command line arguments passed to resperf to standard output.
-h
Print a usage statement and exit.
-i interval
Specifies the time interval between data points in the plot file.
-m max_qps
Specifies the target maximum query rate (in queries per second). This should be higher than the expected maximum throughput of the server being tested. Traffic will be ramped up at a linearly increasing rate until this value is reached, or until one of the other conditions described in the section "Running the test" occurs. The default is 100000 queries per second.
-P plot_data_file
Spefifies the name of the plot data file. The default is resperf.gnuplot.
-r rampup_time
Specifies the length of time over which traffic will be ramped up. The default is 60 seconds.
-L max_loss
Specifies the maximum acceptable query loss percentage for purposes of determining the maximum throughput value. The default is 100%, meaning that resperf will measure the maximum throughput without regard to query loss.

AUTHOR

Nominum, Inc.

SEE ALSO

dnsperf(1)