Benchmarking ExamplesNow that you are familiar with PEAR's Benchmark suite and have looked at ways you can extend it to address specific needs, let's apply those skills to some examples. Mastering any technique requires practice, and this is especially true for benchmarking. Improving code performance through small changes takes time and discipline. The hardest part of productive tuning is not comparing two implementations; the toolset you have built in this chapter is sufficient for that. The difficulty is often in choosing good alternatives to test. Unfortunately, there is no Rosetta stone that will always guide you to the optimal solution; if there were, benchmarking would be a pointless exercise. Realizing potential solutions comes from experience and intuition, both of which only come from practice. In the following sections I cover a few examples, but to gain the best understanding possible, I recommend that you create your own. Start with a relatively simple function from your own code library and tinker with it. Don't be discouraged if your first attempts yield slower functions; learning what patterns do not work is in many ways as important in developing good intuition as learning which do. Matching Characters at the Beginning of a StringA common task in text processing is looking at the leading characters of strings. A common practice is to use substr in a non-assigning context to test strings. For example, to extract all the HTTP variables from $_SERVER, you might use this: foreach( $_SERVER as $key => $val) {
if(substr($key, 0, 5) == 'HTTP_') {
$HTTP_VARS[$key] = $val;
}
}
Although substr is a very fast call, repeated executions add up (for example, if it's used to pick elements out of a large array). Surprising as it may seem, I have seen large applications spend a significant portion of their time in substr due to poorly implemented string parsing. A natural choice for a substr replacement in this context is strncmp, which compares the first n characters of two strings. For example, you can use the following to compare substr to strncmp for picking out the SCRIPT_ variables from $_SERVER: function substr_match($arr) {
foreach ($arr as $key => $val) {
if (substr($key, 0, 5) == 'SCRIPT_') {
$retval[$key] =$val;
}
}
}
function strncmp_match($arr) {
foreach ($arr as $key => $val) {
if(!strncmp($key, "SCRIPT_", 5)) {
$retval[$key] =$val;
}
}
}
require "MyBench.inc";
foreach(array('substr_match', 'strncmp_match') as $func) {
$bm = new MyBench;
$bm->run(1000, $func, $_SERVER);
$result = $bm->get();
printf("$func %0.6f\n", $result['mean']);
}
substr_match 0.000482 strncmp_match 0.000406 A 20% speedup is not insignificant, especially on frequently executed code. Why is substr so much slower than strncmp? substr has to allocate and write its return value and then perform a comparison; on the other hand, strncmp simply performs a character-by-character comparison of the strings. Although PHP hides all the details of memory management, the cost of allocation is still there. Over many iterations, the cost of allocating the 6 bytes for the substr result adds up. Macro ExpansionsIn this example you will use benchmarking to optimize a custom macro expander. Implementing your own macro language can be useful in a number of different contexts, such as supplying limited scripting facilities in a content-management system or an email template system. You might want to be able to template some text like this: Hello {NAME}. Welcome to {SITENAME}.
Your password for managing your account is '{PASSWORD}'.
And have it expanded to this: Hello George. Welcome to example.com. Your password for managing your account is 'foobar'. You can implement your macros as an associative array of matches and replacements. First, you can pull all the recipient users' relevant information from the database: $result = mysql_query("SELECT * from user_profile where userid = $id");
$userinfo = mysql_fetch_assoc($result);
Then you can merge it with an array of "stock" replacements: $standard_elements = array('SITENAME' => 'example.com',
'FOOTER' => "Copyright 2004 Example.com"
);
$macros = array_merge($userinfo, $standard_elements);
Now that you have your macro set defined, you need a macro substitution routine. As a first implementation, you can take the naive approach and iterate over the macro set, substituting as you go: function expand_macros_v1(&$text, $macroset) {
if ($text) {
foreach ($macroset as $tag => $sub) {
if (preg_match("/\{$tag\}/", $text)) {
$text = preg_replace("/\{$tag\}/", $sub, $text);
}
}
}
}
At the core of the routine is this line, which performs the substitution for each tag on the supplied text: $text = preg_replace("/\{$tag\}/", $sub, $text);
You can implement a simple test to guarantee that all your variations behave the same: require "PHPUnit.php";
require "macro_sub.inc";
class MacroTest extends PHPUnit_TestCase {
public function MacroTest($name) {
$this->PHPUnit_TestCase($name);
}
// Check that macros are correctly substituted
public function testSuccessfulSub() {
$macro_set = array( '/\{NAME\}/' => 'george');
$sample_text = "Hello {NAME}";
$expected_text = "Hello george";
$this->assertEquals($expected_text,
expand_macros($sample_text, $macro_set));
}
// Check that things which look like macros but are not are ignored
function testUnmatchedMacro() {
$macro_set = array( '/\{NAME\}/' => 'george');
$sample_text = "Hello {FOO}";
$expected_text = "Hello {FOO}";
$this->assertEquals($expected_text,
expand_macros($sample_text, $macro_set));
}
}
$suite = new PHPUnit_TestSuite('MacroTest');
$result = PHPUnit::run($suite);
echo $result->toString();
Next, you construct your benchmark. In this case, you can try to use data that represents realistic inputs to this function. For this example, you can say that you expect on average a 2KB text message as input, with a macro set of 20 elements, 5 of which are used on average. For test data you can create a macro set of 20 key-value pairs: $macros = array(
'FOO1' => 'george@omniti.com',
'FOO2' => 'george@omniti.com',
'FOO3' => 'george@omniti.com',
'FOO4' => 'george@omniti.com',
'FOO5' => 'george@omniti.com',
'FOO6' => 'george@omniti.com',
'FOO7' => 'george@omniti.com',
'FOO8' => 'george@omniti.com',
'FOO9' => 'george@omniti.com',
'FOO10' => 'george@omniti.com',
'FOO11' => 'george@omniti.com',
'FOO12' => 'george@omniti.com',
'FOO13' => 'george@omniti.com',
'FOO14' => 'george@omniti.com',
'FOO15' => 'george@omniti.com',
'NAME' => 'George Schlossnagle',
'NICK' => 'muntoh',
'EMAIL' => 'george@omniti.com',
'SITENAME' => 'www.foo.com',
'BIRTHDAY' => '10-10-73');
For the template text, you can create a 2048KB document of random words, with the macros {NAME}, {NICK}, {EMAIL}, {SITENAME}, and {BIRTHDAY} interjected into the text. The benchmark code itself is the same you have used throughout the chapter: $bm = new Benchmark_Iterate;
$bm->run(1000, 'expand_macros_v1', $text, $macros);
$result = $bm->get();
printf("expand_macros_v1 %0.6f seconds/execution\n", $result['mean']);
The code yields this: expand_macros_v1 0.001037 seconds/execution This seems fast, but 100 markups per second is not terribly quick, and you can make some improvements on this routine. First, the preg_match call is largely superfluousyou can just make the replacement and ignore any failures. Also, all the PCRE functions accept arrays as arguments for the patterns' and substitutions' variables. You can take advantage of that as well. You can make your routine look like this: function expand_macros_v2(&$text, &$macroset) {
if ($text) {
preg_replace(array_keys($macroset), array_values($macroset), $text);
}
}
This will work, although you will need to preprocess your macros to turn them into pure regular expressions: function pre_process_macros(&$macroset) {
foreach( $macroset as $k => $v ) {
$newarray["{".$k."}"] = $v;
}
return $newarray;
}
Note If you are feeling especially clever, you can change your SELECT to this: SELECT NAME '/\{NAME\}/', '/\{EMAIL\}/'
FROM userinfo
WHERE userid = $userid
The major disadvantage of this is that you are forced to recode the SELECT whenever columns are added to the table. With the SELECT * query, macros magically appear as the table definition is updated. This gives you a significant (15%) performance benefit, as shown here: $bm = new Benchmark_Iterate;
$bm->run(1000, 'expand_macros_v2', $text, pre_process_macros($macros) );
$result = $bm->get();
printf("expand_macros_v2 %0.6f seconds/execution\n", $result['mean']);
expand_macros_v2 0.000850 seconds/execution
You can squeeze a little more improvement out of your code by trying to take advantage of the structure of your macros. Your macros are not random strings, but in fact are all quite similar to one another. Instead of having to match a regular expression for every macro, you can match them all with a single expression and then look them up by key and use an evaluated replacement expression to perform the replacement: function expand_macros_v3(&$text, &$macroset) {
if ($text) {
$text = preg_replace("/\{([^}]+)\}/e",
"(array_key_exists('\\1', \$macroset)?\$macroset['\\1']:'{'.'\\1'.'}')",
$text);
}
}
At the core of this routine is the following replacement: $text = preg_replace("/\{([^}]+)\}/e",
"(array_key_exists('\\1', \$macroset)?\$macroset['\\1']:'{'.'\\1'.'}')",
$text);
Although this routine is complex looking, the idea behind this code is simple: For everything that looks like a tag (that is, a word contained in braces), you perform an evaluated replacement. (The e at the end of your regular expression means that the substitution is evaluated. That is, instead of substituting the text of the replacement block, we execute it with the eval() function, and the result is used for the replacement.) The evaluated expression checks to see whether the suspected tag is a member of the macro set, and if it is, it performs the substitution. This prevents code that looks like a tag but is not (for example, a JavaScript function) from being replaced with whitespace. The benchmark yields the following: expand_macros_v3 0.000958 seconds/execution This seems strange. The code "improvement" (which does fewer regular expression matches) is slower than the original code! What could the problem be? Unlike Perl, PHP does not have the option to have evaluated substitution expressions be compiled once and executed repeatedly. In Perl this is done with s/$pattern/$sub/eo; the o modifier tells the regular expression to compile $sub only once. PHP allows for similar "compiled" regex capability with the preg_replace_callback() function, but it is a bit awkward to use in many contexts. When you use eval on a code block in PHP, it is parsed, compiled, and then executed. The simpler the code that you are using eval on, the less time must be spent in eval. To minimize the cost of using eval on replacement text on every execution, you can attempt to reduce the code to a single function call. Because the function is compiled as part of the main include compilation, you largely avoid the per-call compile overhead. Here is the evaluated substitution that uses a single helper function: function find_macro($sub, &$macros){
return array_key_exists($sub, $macros)?$macros[$sub]:"{$sub}";
}
function expand_macros_v4(&$text, &$macroset) {
if($text) {
$text = preg_replace("/\{([^}]+)\}/e",
"find_macro('\\1', \$macroset)",
$text);
}
}
You might remember the function tr_replace, which, as its name implies, replaces all occurrences of a given string with a replacement string. Because your token names are fixed, str_replace seems like an ideal tool for your task. You can add it to your benchmark as well: function expand_macros_v5(&$text, &$macroset) {
if($text) {
$text = str_replace(array_keys($macroset),
array_values($macroset),
$text);
}
}
By benchmarking these with your same macro set (20 macros defined, 5 used in the text), you get the results shown in Figure 19.2 on different message body sizes. Figure 19.2. A comparison of the linear growth of the token-matching method with the nonlinear growth of the straight preg_replace method.
So, although str_replace() beats preg_replace() when used in the same way, the PHP 4 token-centric method still comes out ahead by a good margin. This is because the token matcher makes only one match, whereas both the str_replace() and preg_replace() methods perform count(array_keys($macroset)) matches. It is an interesting exercise to find the combination of $macroset size and text size below which it becomes preferable to use the pure str_replace() (PHP5) method. On my system, with documents 4KB in size and smaller, this breaking point was 10 macros. Because you have maintained identical APIs for your expand_macro() implementations, you could even dynamically switch to an optimal implementation based on the size of the macro set, although this would likely be overkill. The reason that you get much more scalable performance out of the later macro substitution methods is that the pure preg_replace() and str_replace() methods both require O(M*N) work, where M is the number of macros and N is the size of the document. This is because both of these methods must scan the entire document, looking for each of the macros. In contrast, the tokenization methods (version 3 and version 4) only need to do O(N) matches (because they only match a single pattern on the document) and then do a series (N at most) of O(1) hash-lookups to determine the substitutions. As the size of the macro set grows smaller, the preg_replace() and str_replace() methods become closer to O(N) in speed, and the cost of calling eval() in the tokenization method becomes more visible. Interpolation Versus ConcatenationInterpolation of variables is a fancy name for expanding their values in a string. When you use this: $name = 'George'; $string = "Hello $name!\n"; you cause the current value of $name ('George') to be interpolated into the string $string, resulting in it being assigned the value "Hello George!\n". At the beginning of this chapter I made a statement that the cost of interpolating variables has dropped in PHP 4.3 and PHP 5. Taking that statement at face value would be contrary to the basic message of this book, so let's write a quick test to divine the truth. Both string concatenation and variable interpolation in strings are language primitives in PHP. Neither requires calling a function, and both can be expressed in a short sequence of operations in the PHP virtual machine. They are extremely fast. For this reason, using a wrapper function to package them up for calling from your benchmarking harness will skew your results heavily. Even using your MyBench class will introduce significant bias because you still have to wrap them in a userspace function. To address this in the best way possible, you can write a wrapper that does all the iterations itself (in a tight loop, with no function calls at all), and then benchmark that: require 'RusageBench.inc';
function interpolated($name, $iter) {
for($i=0;$iter; $i++) {
$string = "Hello $name and have a very nice day!\n";
}
}
function concatenated($name, $iter) {
for($i=0;$iter; $i++) {
$string = "Hello ".$name." and have a very nice day!\n";
}
}
$iterations = 100000;
foreach(array('interpolated', 'concatenated') as $func) {
$bm = new RusageBench;
$bm-run(1, $func, 'george', $iterations);
$result = $bm->get();
printf("$func\tUser Time + System Time: %0.6f\n",
($result[mean][ru_utime] + $result[mean][ru_stime])/$iterations);
}
When you run this under PHP 4.2.3, you get the following: PHP 4.2.3 interpolated User Time + System Time: 0.000016 concatenated User Time + System Time: 0.000006 When you run it under PHP 4.3, you get this: PHP 4.3 interpolated User Time + System Time: 0.000007 concatenated User Time + System Time: 0.000004 So although you see a significant improvement in the performance of interpolation, it is still faster to use concatenation to build dynamic strings. Chapter 20, "PHP and Zend Engine Internals" which looks at the internals of the Zend Engine (the scripting engine at the core of PHP), also investigates the internal implementation difference between internal and user-defined functions. |
