How can we believe the results from webpagetest?

I ran webpage test multiple iterations for some of my application pages and the result numbers are different. For example in first run for one page full load time is 2 secs and in another run its 6 secs. My customer is asking how can we believe these results?

You really need to add more runs to the test before making the conclusion. There are multiple things that occur during the connection between the browser and the server which can make that time vary. I read somewhere that to get at least a decent measure, you need to take the MEDIAN of 100 runs. It’s also important to do this over a period of time. You also want to make sure you do this in different browsers since each browser based on your users since this will likely have a slightly different time.

Also, I’d suggest you look at other metrics rather than just full page load.

I think the most important thing to tell your customer is that they can look at the specific recommendations, and make updates/changes that will absolutely make the end user experience faster–the specific amount (time) will depend on the user’s connection, of course, but taking load time and making it 30% faster will still be an improvement for both users.

Also, you can look at measured elements, such as ‘Fully Loaded Bytes In’ = 655 KB and understand why a given page is taking the amount of time it is to load. Improving your page size by optimizing images, for example, will improve page speed.

Page speed can be affected by server load as well, but you can offload the work that a server does by utilizing a CDN. All of these things work together to make a faster web page, yielding a better user experience overall.

It sounds like you may have found a bottleneck which could be optimized. (A great reason to use webpagetest)

Load time does vary from test to test slightly due to the network but not normally that much. Use the waterfall chart to find the differences between the test. Identify the resource that is taking longer in the 6s test and optimize if possible.

You’re not alone in your frustration with this. Many of the servers that run the tests are on donated hardware, and they’re not always the best. Sometimes your test will run on a fast agent, and then next time a slow one - hence the big differences you’re seeing from test to test.

In addition to the other suggestions here (which are all valid), consider running your own instance. It doesn’t take that long to set up, and you control all the hardware so you will get consistent results.

That’s what we ended up doing (and turned it into a service so others could use it as well).

Guys, your recent updates have WPT messed up completely. NO way to trust this any more. I’ve seen charts, where a download of HTML happend before DNS request and connecting to the server. How “slow” can a server be to make this happen? :-\

Feel free to leave a link to said test. I haven’t encountered anything like that.

See this with some context thoughts. Link to WPT result you’ll find inside the post:

Hi if you have a look at the data plots you can see alot more information.

but yes it appears that you’ve found a bug.