wptdriver always using first browser listed

WebPageTest Private Instances
1

Hi,

Sorry for the long post - i’m trying to provide all the relevant info!

I’m using the EC2 agent image (ami-8b2c2fff) with the location and url set to my WPT-Private instance server.

If I use the web interface to run a test I can choose any of IE, Firefox, Chrome or Safari and they all work.

If I try running tests using the API and a script (via etsy’s WPT-Script) then it recognises the 4 browsers available at the site, and says it’s running a test for each browser. IE runs (via URLBlast) but the wptdriver will always run 3 tests against the same browser.

The browser it uses is determined by the order they’re listed in locations.ini.

wptdriver.ini looks like this:
[WebPagetest]
url=http://xx.xxx.x.xx/
ec2=1
location=ec2-eu-west_wptdriver
Time Limit=60
software=http://www.webpagetest.org/installers/software.dat

[Chrome]
exe=“C:\Program Files\Google\Chrome\Application\chrome.exe”
options=‘–load-extension=“%WPTDIR%\extension” --user-data-dir=“%PROFILE%” --no-proxy-server’
installer=http://www.webpagetest.org/installers/browsers/chrome.dat

[Firefox]
exe=“C:\Program Files\Mozilla Firefox\firefox.exe”
options=‘-profile “%PROFILE%” -no-remote’
installer=http://www.webpagetest.org/installers/browsers/firefox.dat
template=firefox

[Safari]
exe=“C:\Program Files\Safari\safari.exe”

Locations.ini:
[locations]
1=ec2-eu-west
default=ec2-eu-west

[ec2-eu-west]
1=ec2-eu-west-IE9
2=ec2-eu-west_wptdriver
default= ec2-eu-west_wptdriver
label=“ec2-eu-west, Dublin, IE9,Chrome,Firefox,Safari”

[ec2-eu-west-IE9]
browser=IE 9
label=“EU West IE9”

[ec2-eu-west_wptdriver]
browser=Firefox,Chrome,Safari
label=“EU West”

http://server/install shows this:
ec2-eu-west : ec2-eu-west, Dublin, IE9,Chrome,Firefox,Safari
ec2-eu-west-IE9 : EU West IE9 - 1 agents connected
ec2-eu-west_wptdriver : EU West - 1 agents connected

my wpt-script is:

"server": "http://server",
"pending_dir": "private",
"script": "myaccount_stage.wpt",
"logging_ns": "webpagetest.private",
"splunkLog": "/var/log/webpagetest/results.log",
"run_options": {
    "video": 1,
    "url": "http://stage.site.com/login",
    "location" : "ec2-eu-west:Chrome",
    "runs": 2,
    "private": 0
}

(I’ve also tried “ec2-eu-west_wptdriver:Chrome”, “ec2-eu-west_wptdriver”, etc. in the location)

but it always shows:
2014-05-29T21:56:43+01:00 Started test with config: /var/lib/wpt-script/bin/…/conf/stage.conf. Testing myaccount_stage.wpt across 4 locations.
2014-05-29T21:56:43+01:00 Started signedout test for scripted test / ec2-eu-west-IE9 / IE 9
2014-05-29T21:56:43+01:00 Started signedout test for scripted test / ec2-eu-west_wptdriver / Firefox
2014-05-29T21:56:43+01:00 Started signedout test for scripted test / ec2-eu-west_wptdriver / Chrome
2014-05-29T21:56:43+01:00 Started signedout test for scripted test / ec2-eu-west_wptdriver / Safari

Regardless of saying it’s going to run across all 4 browsers, the test always runs against IE and the first browser listed in wptdriver in locations.ini.

http://server/getLocations.php shows:
Location Idle Testers Total Tests Being Tested High Priority P1 P2 P3 P4 P5 P6 P7 P8 P9
ec2-eu-west-IE9 0 3 1 0 0 0 0 0 2 0 0 0 0
ec2-eu-west_wptdriver:Firefox 0 11 1 0 0 0 0 0 10 0 0 0 0
ec2-eu-west_wptdriver:Chrome 0 11 1 0 0 0 0 0 10 0 0 0 0
ec2-eu-west_wptdriver:Safari 0 11 1 0 0 0 0 0 10 0 0 0 0

So, 2 problems - It’s ignoring what I say in locations, and wptdriver runs the same browser, regardless of knowing it has 3.

Any help appreciated!

Thanks

Rich

2

you should be using ec2-eu-west_wptdriver:Chrome as your location when you run a test.

If you’re running the EC2 AMI’s you should also not be touching the configs (don’t need to edit wptdriver.ini) - it’s all configured through user data when you spin up an instance.

If you run tests through the UI do they work? I’d get them working there first and then add the scripting on top of it to help keep the possible broken pieces manageable.

3

Hi,

Sorry for the original post. I have managed to narrow down the problem in the past 24 hours. I’ve got the wpt-script part working and requesting the correct locations. I am now just left with the same browser always being picked by wptdriver.

Here’s a truncated example of a runtest request:

"GET /runtest.php?runs=2&fvonly=1&sensitive=1&private=0&f=xml&notify=webpagetest%40etsy.com&video=1&locations%5B0%5D=ec2-eu-west_wptdriver%3ASafari&urlETCETC

As you can see - it’s requesting Safari, but wptdriver is using chrome instead.

With regards to changing wptdriver.ini - i populated the userdata as required, urlBlast.ini picked up the info but wptdriver.ini didn’t.

I’m sure i’d read a post of the forum saying this was a known problem (but happy to be corrected on that).

What I really don’t understand is that the wptdriver browser being used is determined by the order the browsers are listed in locations.ini, not wptdriver.ini.

Thanks

Rich

4

locations%5B0%5D=ec2-eu-west_wptdriver%3ASafari is malformed (and incorrect).

It should be location=ec2-eu-west_wptdriver%3ASafari

What’s happening is that it doesn’t think a location is specified and it is picking the default.

5

OK,

Found that problem. now the request is:

GET /runtest.php?runs=2&fvonly=1&sensitive=1&private=0&f=xml&notify=someone%40here.com&video=1&location=ec2-eu-west_wptdriver&url=BLAHBLAHBLAH&browser=Safari

But it’s still using the first browser defined in locations.ini
–snip—
[ec2-eu-west_wptdriver]
browser=Firefox,Chrome,Safari
label=“EU West”
–snip–

Here’s my wptdriver.ini:
[WebPagetest]
url=http://server/
ec2=1
location=ec2-eu-west_wptdriver
Time Limit=60
software=http://www.webpagetest.org/installers/software.dat

[Firefox]
exe=“C:\Program Files\Mozilla Firefox\firefox.exe”
options=‘-profile “%PROFILE%” -no-remote’
installer=http://www.webpagetest.org/installers/browsers/firefox.dat
template=firefox

[Chrome]
exe=“C:\Program Files\Google\Chrome\Application\chrome.exe”
options=‘–load-extension=“%WPTDIR%\extension” --user-data-dir=“%PROFILE%” --no-proxy-server’
installer=http://www.webpagetest.org/installers/browsers/chrome.dat

[Safari]
exe=“C:\Program Files\Safari\safari.exe”

6

Closer, but still not right. The browser needs to be included with the location - the problem before was that it was “locations” instead of “location” and the = was urlencoded. Should look like:

GET /runtest.php?runs=2&fvonly=1&sensitive=1&private=0&f=xml&notify=someone%40here.com&video=1&location=ec2-eu-west_wptdriver%3ASafari&url=BLAHBLAHBLAH

7

Hi,

Thanks for your help with this. I think we’re there now - there appears to be a bug in the wpt-script which we’ve made a fix for (essentially, as you could see it was supplying the browser as a separate option rather than appended to the location.)

I’m not sure yet if this doesn’t break some default behaviour so I won’t submit it back at this point.

But for anyone else who may find this helpful:

around line 190 of WebpagetestClient.php do:
//Stop location + browser being 2 options
// $options[‘location’] = $location[‘location’];
// $options[‘browser’] = $location[‘browser’];
//Replace it so it’s a single location option.
$options[‘location’] = $location[‘location’].“:”.$location[‘browser’];

And on line 208:
// $this->log(“Started $login_string test for $url / {$options[‘location’]} / {$options[‘browser’]}”);
// $options[‘browser’] doesn’t exist anymore
$this->log(“Started $login_string test for $url / {$options[‘location’]} / {$location[‘browser’]}”);

Thanks again for your help!