… and they stated … “We have noticed specifically webpagetest.org have inconsistent nodes that provide results that differ from other more reputable monitoring tools…”.
Would you agree with their statement about the inconsistent nodes?
Yes and no. The nodes are not necessarily directly comparable with each other because they are on different hardware and different connectivity but any given node should be quite consistent. In particular, the Dulles nodes are about as good as it gets.
The performance won’t be as FAST as running from a backbone agent but that is by design and fully expected.
Is the issue with the overall time for some of the resources or the first byte times? The overall times for some of the static resources looks like they are bandwidth constrained at 1.5Mbps which is why some of the blue bars are so long and it’s a function of the size of each resource and the number of them being downloaded at the same time. The bandwidth graph at the bottom looks like it capped out for most of the time between 1 and 2.4 seconds.
In the advanced settings you can select the type of connectivity you would like to simulate. The native connectivity is a lot faster and then we use dymmynet to make it look like a realistic end-user connection.
You’re welcome to use any profile you’d like but one of the biggest values that WebPagetest brings is the more-realistic end-user connectivity. If you test from datacenter nodes you usually have < 1ms RTT and 1GBps connectivity to CDN edge nodes which the CDNs sometimes colocate in the same datacenters (and even racks) as the testing company’s equipment. That results in unrealistic gains in performance when measuring CDN impact.
Question, what do you mean by “running a flat-out”? And how does one determine where the bandwidth caps out at? Does the graph look to be capped out at 3.1 seconds also then?
Sorry, the “DSL” configurations cap bandwidth at 1.5Mbps. Measuring bandwidth on the client side can be a bit spiky so it’s not unusual to see peaks over that but the scale on the bandwidth chart says it is 0-2.5Kbps so just over half way would be the 1.5Mbps limit.
From 1s-2.4s the bandwidth line is hovering right around where I’d expect 1.5Mbps to be on the graph.
Running flat-out means 0 added latency and no bandwidth limit (custom profile with 0 for all of the settings).
The top of the graph is 2.5Mbps and the graph is broken into 4 bands (grey/white alternating) so each chunk is roughly 631Kbps. 1/2 way up the graph would be 1.2Mbps, etc. Nothing scientific, mostly just eyeballing it.