Trouble with WPT Server config

Having some trouble with my WPT server config. I can see via the apache logs that my agents are polling the server properly:

1.1.1.1 - - [20/Feb/2011:10:55:03 +0000] “GET /work/getwork.php?video=1&location=Vancouver&key=mykey1&pc=vancouver01&ver=261 HTTP/1.1” 200 - “-” “urlBlast”
2.2.2.2 - - [20/Feb/2011:10:55:04 +0000] “GET /work/getwork.php?video=1&location=Sydney&key=mykey2&pc=sydney01&ver=261 HTTP/1.1” 200 - “-” “urlblast”

However, they never seem to get the work job in the response. My locations.ini is as follows:

[locations]
1=Vancouver_loc
2=Sydney_loc
default=Vancouver_loc

[Vancouver_loc]
1=Vancouver
default=Vancouver
label=Vancouver, BC

[Sydney_loc]
1=Sydney
default=Sydney
label=Sydney, AUS

[Vancouver]
browser=IE 7
latency=0
label=“Vancouver, BC”
key=mykey1

[Sydney]
browser=IE 7
latency=0
label=“Sydney, AUS”
key=mykey1

When you submit the test on the server side does it go to the progress screen and say it is waiting behind X tests? If so, can you check the work/jobs/Vancouver and work/jobs/Sydney directories on the server to see if the job files are there?

Some possible causes that come to mind:

  • The apache user doesn’t have write permissions to the work/jobs directory
  • The tmp directory under the docroot couldn’t be created or the apache user doesn’t have write permissions to it (the lock files are stored in there)
  • dummynet isn’t installed on the test machine so it’s fetching the jobs but they are failing right away

Thanks Patrick,

Looks like the docroot/tmp directory wasn’t created, even though I chowned my docroot to apache. I manually created the docroot/tmp directory with apache permissions and everything’s running properly now. Thanks again!

PS. Maybe include an empty tmp dir in your package.

Yep, good idea. The tmp directory was a last-minute change that I made the day of the release to try and be better about consolidating writable directories (and allow for the frequently accessed temp files to be put on tmpfs).

Patrick,

Seem to have another 2 issues:

  1. Summary preview of waterfall/optimization works, but the detailed 1_waterfall.png or 1_optimization.png are missing.
  2. Video doesn’t seem to work, it just keeps saying waiting for video.

1 - Are all of the rewrite rules in place? The same logic is used to build the thumbnail images as the full waterfall so the only reason it should fail is if the requests aren’t being rewritten properly (there aren’t actual png files on the server anymore, they’re generated dynamically by waterfall.php).

2 - You need to install avisynth on at least one of the test machines (the installer should be in the bundle). It’s needed to create the videos from the test captures.

Thanks,

-Pat

I recently created a private instance and also bumped into the issue with videos not being created, even with Avisynth installed etc.
Solution for me was: create the dir /work/video/
And that did the trick.

Patrick > can you fix this in the build? Txs.

There was some weirdness in the .htaccess file I had. I reinstalled everything from scratch and noticed that results directory had to be created as well as tmp. Thanks again. I still seem to have trouble creating videos, the filmstrip thumbnails work but I can’t create videos for some reason.

Do you see a work/video directory? If not, can you create it? Also, did you install AviSynth on the test machine?

I’ll make sure the next release has all of the correct folders set up before-hand (there are still more directories than I’d like that need to be configured with write permissions :frowning: ).

Thanks,

-Pat

work/video dir exists with a tmp subdir. AviSynth installed on test machines. Video used to work when I was using an older version of webpagetest. I actually have another two oddities, some webpages don’t seem to analyze text compression. I see a N/A and no details. I suspect it may be to do with SSL sites only? Have’t confirmed yet. Also, Test History doesn’t show anything at all for some reason.

N/A for text compression is SSL-specific. The lower layers of IE strip the gzip headers and if I change the behavior then IE goes into a mode where it waits for the full response before processing (which breaks early flush testing).

Are you requesting the tests as private? That would prevent them from showing in the logs. Also make sure the logs folder exists and is writable.

Can you send a couple of entries from your access log for the getwork.php requests (looking for a video=1 flag)?

Thanks,

-Pat

I do have test as private on. I turned them off and it still doesn’t seem to show. I don’t recall having this problem with previous versions of WPT.

I’ll send logs for getwork.php later.

Check the logs directory to make sure it’s writable and see if there are log files showing up - that’s where the test history is written (one file per day).

Still don’t seem to get compression info with SSL with private unchecked. On public WPT however, compression shows up.

RE: Video
x.x.x.x - - [15/Mar/2011:08:31:07 +0000] “GET /work/getwork.php?video=1&location=Sydney&key=*******&pc=AUS&ver=261 HTTP/1.1” 200 - “-” “urlBlast”

In results folder I see this under video_1 subfolder but WPT never shows the video:

-rw-rw-rw- 1 apache apache 3487 Mar 15 04:34 frame_0000.jpg
-rw-rw-rw- 1 apache apache 35497 Mar 15 04:34 frame_0018.jpg
-rw-rw-rw- 1 apache apache 36219 Mar 15 04:34 frame_0020.jpg
-rw-rw-rw- 1 apache apache 36219 Mar 15 04:34 frame_0021.jpg
-rw-rw-rw- 1 apache apache 36474 Mar 15 04:34 frame_0023.jpg
-rw-rw-rw- 1 apache apache 36474 Mar 15 04:34 frame_0028.jpg
-rw-rw-rw- 1 apache apache 36474 Mar 15 04:34 frame_0029.jpg
-rw-rw-rw- 1 apache apache 36475 Mar 15 04:34 frame_0032.jpg
-rw-rw-rw- 1 apache apache 36475 Mar 15 04:35 frame_0035.jpg
drwxrwxrwx 3 apache apache 4096 Mar 15 04:35 …
drwxrwxrwx 2 apache apache 4096 Mar 15 04:35 .
-rw-rw-rw- 1 apache apache 590 Mar 15 04:54 video.avs
-rw-rw-rw- 1 apache apache 228 Mar 15 04:54 stats.avs
-rw-rw-rw- 1 apache apache 200 Mar 15 04:54 readme.txt

Compression detection doesn’t work with SSL (private install or on the public server) - it’s a limitation on the information I have available from IE.

It looks like you have the necessary components to create the videos but something is going wrong with the rendering (or posting back). Do you have an entry in the access log for workdone.php shortly after the getwork.php entry?

If you look in the results/video directory, do you see any folders created?

Next step in debugging is going to be to turn on debugging in urlblast (add debug=1 to urlblast.ini) and run again. It will log information about the attempt to create the video (it downloads a zip file, extracts it and then runs x264.exe on it to create the video file before posting it back to the server).

Sorry it’s so frustrating, there are a lot of moving pieces in the video creation path (been considering simplifying it so I can do it all in php on the server but avisynth provides a lot of flexibility that I’d hate to give up).

Thanks,

-Pat