Load Generators
Having to wait for a condition to manifest itself on a live site is not an efficient method to collect statistics on pages. In many cases it might be impractical to do in-depth diagnostics on a production server. In other cases you might need to generate load in excess of what the site is currently sustaining.
To tackle this problem of being able to supply traffic patterns on demand, you can use load generators. Load generators come in two flavors: contrived and realistic. A contrived load generator makes little effort to generate traffic patterns akin to a normal user; instead, it generates a constant and unforgiving request pattern against a specific page or pages.
Contrived load generators are very useful for testing a specific page but less useful when you're attempting to identify overall site capacity or obscure bottlenecks that appear only under real-world conditions. For those, you need a realistic load generatoroften known as a playback tool because a realistic load generator tends to work by reading in traffic patterns from a log file and then playing them back as a timed sequence.
ab
The simplest of the contrived load generators is ApacheBench, or ab, which ships as part of Apache. ab is a simple multithreaded benchmarking tool that makes a number of requests with specified concurrency to a given URL. Calling ab "simple" probably does not do it justice because it is a robust tool that has a number of nice features.
Here is a sample run against my Web log, in which I've specified 1,000 requests with a concurrency of 100 requests:
> /opt/apache/bin/ab -n 1000 -c 100 http://localhost/~george/blog/index.php
This is ApacheBench, Version 1.3d <$Revision: 1.65 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
Benchmarking www.schlossnagle.org (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests
Server Software: Apache/1.3.26
Server Hostname: www.schlossnagle.org
Server Port: 80
Document Path: /~george/blog/index.ph
Document Length: 33086 bytes
Concurrency Level: 100
Time taken for tests: 41.792 seconds
Complete requests: 1000
Failed requests: 0
Broken pipe errors: 0
Non-2xx responses: 0
Total transferred: 33523204 bytes
HTML transferred: 33084204 bytes
Requests per second: 23.93 [#/sec] (mean)
Time per request: 4179.20 (mean)
Time per request: 41.79 (mean, across all concurrent requests)
Transfer rate: 802.14 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 38 92.6 1 336
Processing: 585 3944 736.9 4066 10601
Waiting: 432 3943 738.1 4066 10601
Total: 585 3982 686.9 4087 10601
Percentage of the requests served within a certain time (ms)
50% 4087
66% 4211
75% 4284
80% 4334
90% 4449
95% 4579
98% 4736
99% 4847
100% 10601 (last request)
I averaged almost 24 requests per second, with an average of 41.79 milliseconds taken per request, 39.43 of which was spent waiting for data (which corresponds roughly with the amount of time spent by the application handling the request).
In addition to the basics, ab supports sending custom headers, including support for cookies, HTTP Basic Authentication, and POST data.
httperf
When you need a load generator with a broader feature set than ab, httperf is one tool you can use. httperf was written by David Mosberger of Hewlett Packard Research Labs as a robust tool for measuring Web server performance. It was designed for high-volume throughput, full support for the HTTP 1.1 protocol, and easy extensibility. These latter two features are its significant distinguishers from ab. If you need to test behavior that requires Content-Encoding or another HTTP 1.1specific option, httperf is the tool for you.
To perform an httperf run similar to the ab run in the preceding section, you would use this:
> httperf --client=0/1 --server=localhost --port=80 --uri=/~george/blog/index.php
--rate=40 --send-buffer=4096 --recv-buffer=16384 --num-conns=100 --num-calls=1
Total: connections 1000 requests 1000 replies 1000 test-duration 50.681 s
Connection rate: 19.7 conn/s (50.7 ms/dconn, <=421 concurrent connections)
Connection time -: min 274 avg 8968 max 33513 median 6445 stddev 6340
Connection time -: connect 2596.0
Connection length [replies/conn]: 1.000
Request rate: 19.7 req/s (50.7 ms/req)
Request size [B]: 93.0
Reply rate [replies/s]: min 1.2 avg 19.8 max 25.8 stddev 8.4 (10 samples)
Reply time -: response 6110.0 transfer 262.8
Reply size [B]: header 460.0 content 33084.0 footer 2.0 (total 33546.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0
CPU time [s]: user 0.64 system 13.71 (user 1.3% system 27.1% total 28.3%)
Net I/O: 648.2 KB/s (5.3*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
One of the nice features of httperf is its support for multiple work-load generators. The default generator showcased in this example is the fixed URL generator because it load-tests based on a single fixed URL. Additional generators include the log-based generator, the session simulator, and the realistic data generator.
The Log-Based Generator
The log-based generator is used to send requests to a series of URLs specified in a file. You specify file by using -wlog=loop,file. loop is a y/n value that specifies whether httperf should loop back to the beginning of the file when it reaches the end. If -uri is specified, it will be prepended to all URLs. Here is an example that reads URLs from the log /tmp/urllist:
httperf --client=0/1 --server=www.schlossnagle.org --port=80
-wlog=y,/tmp/urllist --rate=40 --send-buffer=4096
--recv-buffer=16384 --num-conns=100 --num-calls=1
The items specified in the URL list need to be delimited by ASCII nulls (chr(0)).
The Session Simulator
The session simulator attempts to simulate a user's behavior. You set the simulator's behavior by using four parameters: N1, N2, X, and L. A session consists of N2 calls. The calls are broken up into bursts of L calls as follows: the initial request is made, and when it returns completely, all the subsequent requests are issued concurrently. This is designed to represent the loading of a Web page with L 1 images / secondary objects: the initial page is requested, and when its HTML has returned, all the images on the page are loaded. The session then pauses for X seconds before starting its next burst.
N1 specifies the number of sessions to initiate. The parameters are specified with the following syntax:
--wsess=N1,N2,X -burst-length=L
The Realistic Data Generator
httperf also supports pseudorealistic playback of user sessions via a simple scripting mechanism. A sample playback script for going to the a php.net mirror, reading the page for 10 seconds, and clicking through to the docs page looks like this:
/index.php think=10
//images/016/news/afup-logo.gif
//images/016/news/chmhelp.gif
//images/016/news/conference_php_quebec.gif
//images/016/news/hu_conf.gif
//images/016/news/international_conference_2003_spring.gif
//images/016/news/mysqluc2003.png
//images/016/news/phpcon_logo.png
//images/016/php_logo.gif
//images/016/rss10.gif
//images/016/spacer.gif
/backend/mirror.gif
/docs.php
//images/016/php_logo.gif
//images/016/spacer.gif
Each outdented line denotes the beginning of a burst; the indented lines below them are subrequests in the burst. Each burst-initiation line can have individualized settings to indicate the pause time between itself and the next burst, changing the method, setting POST data, and so on.
The ability to script sessions is incredibly powerful; but the format for the scripting files, although elegant, makes translating real-world sessions into scripts difficult. It would be preferable to have a tool that can read actual Apache logs and replay them not only element-by-element but in the same timing separation as in the original request. Daiquiri fits this need.
Daiquiri
Daiquiri is a Web workload generator that understands Common Log Format Apache logs and replays them. Daiquiri locates its options in a configuration file of the following form:
Schema test = {
Headers = "Host: www.schlossnagle.org\r\n"
Log = "/var/apache-logs/replay.log"
RequestAllocation "reqalloc.so::SingleIP" => {
192.168.52.67:80
}
ChunkLength = 5
ChunkCushion = 1
HTTPTimeout = 200
MultiplicityFactor = 1
}
Headers specifies a string of arbitrary headers, separated by new lines.
Log specifies the logfile to be read back from. The log must be in common log format.
RequestAllocation specifies how the requests are to be made. Daiquiri supports dynamic loading of request modules, and this is handy if the stock modes do not satisfy your needs. There are two modes built as part of the source distribution:
ChunkLength and ChunkCushion specify how far in advance the logfile should be parsed (in seconds). Daiquiri assumes that the logfile lines are in chronological order.
Setting MultiplicityFactor allows additional traffic to be generated by scheduling each request multiple times. This provides an easy way to do real-time capacity trending of Web applications with extremely realistic data.
| 