multiple test runs and "Test Error: Data is missing."

WebPageTest Private Instances
1

Hi,

I have a WPT private instance running connected to 2 AWS EC2 m1.medium instances in the AWS East region using the following AMIs:

ie11-20140218 (ami-4f202626)
ie9-20121031 (ami-c262d9ab)

when I run a test with multiple runs, it more often than not results in a “Test Error: Data is missing.” message for one of the tests. It also does it for single run tests as well.

Is there a way for me to debug where the problem may be occurring?

These are the pings from the 2 agents:

x.x.x.x - - [05/Jun/2014:14:10:38 -0700] "GET /work/getwork.php?shards=1&location=US_E_wptdriver&key=xxx&software=wpt&version=2.13.0.160&ver=160&pc=xxx&ec2=xxx&dns=xxx&freedisk=7.164&GPU=0 HTTP/1.1" 200 368 "-" "WebPagetest Driver" x.x.x.x - - [05/Jun/2014:14:10:40 -0700] "GET /work/getwork.php?location=US_E_IE9_wptdriver&key=xxx&software=wpt&ver=78&pc=xxx&ec2=xxx&freedisk=14.680 HTTP/1.1" 200 368 "-" "WebPagetest Driver"

here is my /install page
[attachment=393]

2

Try updating the agents: https://sites.google.com/a/webpagetest.org/docs/private-instances#TOC-Updating-Test-Agents

The 2.13.0.160 agents are pretty old and there have been a lot of fixes since. There have also been a lot of fixes on the server side (both with 2.14 and even since then) that should reduce the incidence significantly. I’ll see if I can cut a 2.15 release in the next week or two with all of the latest and greatest.

3

thanks Patrick. I updated the agents as well as pulling the latest from github and multiple runs seems to work better. I still had a few multi-test runs (3 tests per run) that had at 1 test fail, but better than before.

I noticed while tailing my access & error logs that I get random HTTP 403s for resultimage.php, eg. look at the 8th request for resultimage.php in this snippet from the access_log:

x.x.x.x - - [06/Jun/2014:10:39:36 -0700] "POST /work/resultimage.php HTTP/1.1" 200 370 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:36 -0700] "POST /work/resultimage.php HTTP/1.1" 200 369 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:36 -0700] "POST /work/resultimage.php HTTP/1.1" 200 370 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:36 -0700] "POST /work/resultimage.php HTTP/1.1" 200 369 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:37 -0700] "POST /work/resultimage.php HTTP/1.1" 200 369 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:37 -0700] "POST /work/resultimage.php HTTP/1.1" 200 369 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:37 -0700] "POST /work/resultimage.php HTTP/1.1" 200 369 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:37 -0700] "POST /work/resultimage.php HTTP/1.1" 403 447 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:37 -0700] "POST /work/resultimage.php HTTP/1.1" 200 369 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:38 -0700] "POST /work/resultimage.php HTTP/1.1" 200 370 "-" "WebPagetest Driver" x.x.x.x - - [06/Jun/2014:10:39:38 -0700] "POST /work/workdone.php HTTP/1.1" 200 470 "-" "WebPagetest Driver"

the apache error_log entry for this 403 is

[Fri Jun 06 10:39:37 2014] [error] [client x.x.x.x] client denied by server configuration: /x/public_html/webpagetest/www/work/resultimage.php

i actually have quite a lot of these in the error log now that I grepped through it. Any idea what this could be? The first thing I thought was maybe an htaccess issue, but that wouldn’t explain why it would 403 for only some requests. I have overrides enabled for this vhost & the rewrites are working as far as I can tell:

<Directory /...../webpagetest/www> Options -Indexes IncludesNOEXEC SymLinksifOwnerMatch allow from all AllowOverride All </Directory>

(i’m on Apache/2.2.22 if that matters)

thanks,
-norm

4

Maybe check the configs for POST body size and temp directoy for POSTs to make sure the web server user has permissions. Those are 2 config issues I have seen.

5

And by the configs, I mean the Apache ones, not the PHP ones.

6

i know i’m resurrecting an old thread, but i just ran across this issue again today and just wanted to closed the loop as to the root cause – ends up I forgot I had the apache module mod_evasive installed, which prevents DoS-ing and the quick successive calls to resultimage.php after a test was tripping mod_evasive to return 403 requests. i whitelisted the tester IPs and all is well again.