Different results using site & API

WebPageTest Private Instances
1

Hi,

I’ve recently been experimenting setting up scripts to automate tests in the hosted platform and for a particular test got significantly different results between the automated script and manually going to webpagetest.org and running the test. I don’t think it is an issue with the script as I only found this with one particular test. The tests are for the same URL, same location, only using first view.

Has anyone else experienced something like this?

2

Were you running the automated script against WebPagetest or your own instance? Also, do you see it regularly or was it a one-time problem?

The API and the UI both use the exact same code path but it’s possible one of the options was set differently in the UI (assuming both were run against WebPagetest for the same location, browser and connectivity).

Thanks,

-Pat

3

Hi Pat,

Thanks for the response.

Both tests are using the public Dulles_IE7 instance, but I get the difference no matter which location I use.

If you compare the two results below, as far as I can tell, the settings for each are identical, the only difference I can see is that the first was submitted using the API, second using the web interface.

http://www.webpagetest.org/result/101115_c301b1873b7f9a4fc4afe83066344ee6/
http://www.webpagetest.org/result/101115_4e6ad601b1de2f59c0b123e02e33d1e2/

Can you see something that I am missing?

Thanks,
Mark.

4

The urls are different. The API is missing the &Nights=7. My guess is that you’re not url-encoding the url before submitting :slight_smile:

Thanks,

-Pat

5

Hi Pat,

Thanks for following that up, I have replaced those ampersands now, looks to be working well.

I spent some time yesterday modifying the script we use to allow for the connection speed to be passed in as variable, expecting that if I passed in the same profile as Dulles_IE7 standard, the results would be the same.

Can you see any reason why the two tests down the bottom would return such different results? The first returns ~5secs, the second ~12 seconds. My expectation would be that they would be practically identical.

http://www.webpagetest.org/runtest.php?f=xml&url=cnn.com&location=Dulles_IE7.custom&bwIn=1500&bwOut=384&latency=50&plr=0&runs=10&private=1&fvonly=1

http://www.webpagetest.org/runtest.php?f=xml&url=cnn.com&location=Dulles_IE7&runs=10&private=1&fvonly=1

Thanks again,
Mark.

6

Sorry, mistake in the documentation - it’s bwDown and bwUp (instead of bwIn and bwOut). I just fixed the docs. It should be identical once it’s working because the standard profiles are just pre-configured settings. Everything is effectively a custom profile.

Thanks (and sorry about that),

-Pat

7

No apologies necessary, appreciate the help.

You have a great tool here. Having lots of fun playing with it!

Thanks again,
Mark.