Benchmarks

Greetings everyone,

I’m currently running a private instance, and exploring benchmarks. Going into /benchmarks/ brings up the docTime graphs of the benchmarks that I have running. when I click on a data point two options are available:

[list]
[]Scatter Plot
[
]Comparison Distribution
[/list]

Scatter Plot works great and it offers a ton of valuable information. Comparison distribution is a different story though. It leads me to /benchmarks/delta.php. There is supposed to be a graph on this page, which does not show. Digging through the code I end up in /benchmarks/data.inc.php, on function [color=#32CD32]LoadDelta[/color]. In the forEach loop found in this function, there are two lines of code that read like this:

[php]foreach ($median_data[$config][$loc][$cached] as $url => &$data) {
if (array_key_exists($url, $median_data[$ref][$refLoc][$cached])) {[/php]

This piece of code is failing because it tries to find the url from config::loc::cached in ref::refloc::cached but the function [color=#32CD32]loadMedianData[/color] never creates such structure. [color=#32CD32]LoadMedianData [/color]creates an associative array containing the parsed results of the benchmark. Let me provide an example:

Lets say we run a benchmark with two configurations:[color=#32CD32] ‘SiteA’[/color] and ‘[color=#32CD32]SiteB[/color]’. Site A is running from location ‘[color=#32CD32]Office_IE9.Cable[/color]’ and it has 3 urls: [color=#32CD32]www.mysitea.com[/color], [color=#32CD32]www.mysitea.com/my_video_page[/color], and [color=#32CD32]www.mysitea.com/some_page[/color]. Site B is running from ‘[color=#32CD32]Office_wptdriver:Chrome.3G[/color]’ and it has also 3 urls: [color=#32CD32]www.mysiteb.com[/color], [color=#32CD32]www.mysiteb.com/my_profile_page[/color], and [color=#32CD32]www.mysiteb.com/some_lost_page[/color] .

After succesful completion of the benchmark, function LoadMedianData retuns an array that looks something like this:

[php]
$median_data[‘SiteA’][‘Office_IE9.Cable’][0][‘http://www.misitea.com/’] = associative_results_array()
$median_data[‘SiteA’][‘Office_IE9.Cable’][0][‘404错误!’] = associative_results_array()
$median_data[‘SiteA’][‘Office_IE9.Cable’][0][‘404错误!’] = associative_results_array()
$median_data[‘SiteB’][‘Office_wptdriver:Chrome.3G’][0][‘http://www.misiteb.com/’] = associative_results_array()
$median_data[‘SiteB’][‘Office_wptdriver:Chrome.3G’][0][‘http://www.misiteb.com/my_profile_page’] = associative_results_array()
$median_data[‘SiteB’][‘Office_wptdriver:Chrome.3G’][0][‘http://www.misiteb.com/some_lost_page’] = associative_results_array()
[/php]

Following this example, when execution arrives to:

[php]foreach ($median_data[$config][$loc][$cached] as $url => &$data) {
if (array_key_exists($url, $median_data[$ref][$refLoc][$cached])) {[/php]

It tries to run something like this:

[php]foreach ($median_data[‘SiteA’][‘Office_IE9.Cable’][0] as $url => &$data) {
if (array_key_exists(‘http://www.misitea.com/’, $median_data[‘SiteB’][‘Office_wptdriver:Chrome.3G’][0])) {[/php]

This statement fails all the time, and the function always retuns NULL, thus no graph :frowning:

My questions are: What is the purpose of delta.php? I mean, if I know what is supposed to do then I can ‘fix’ it. I’m I missing something in the setup of my benchmarks? No mean to insult, but is the code possibly faulty?

Thank you in advance for your answers.

Cheers,

Sorry, you’re wandering into stuff that hasn’t been cleaned up for general consumption yet. The way the benchmarks are set up currently, it basically expects the same set of URLs across various configurations and lets you compare the configurations against each other (which is why you’re having issues with delta.php).

Think “mod_pagespeed accelerated vs Not Accelerated” or “Chrome Stable vs Chrome Canary”. I haven’t had a chance to focus on also having it support general performance trending and things more like “Us vs Competitor”.