We are currently using Edgecast CDN and I had opened a ticket with them about static file download times.
http://www.webpagetest.org/result/120717_J9_1229dc79d0f7153a5d5e47c26275b47f/1/details/
… and they stated … “We have noticed specifically webpagetest.org have inconsistent nodes that provide results that differ from other more reputable monitoring tools…”.
Would you agree with their statement about the inconsistent nodes?
Let me know your thoughts.
Regards,
TRX
Yes and no. The nodes are not necessarily directly comparable with each other because they are on different hardware and different connectivity but any given node should be quite consistent. In particular, the Dulles nodes are about as good as it gets.
The performance won’t be as FAST as running from a backbone agent but that is by design and fully expected.
Is the issue with the overall time for some of the resources or the first byte times? The overall times for some of the static resources looks like they are bandwidth constrained at 1.5Mbps which is why some of the blue bars are so long and it’s a function of the size of each resource and the number of them being downloaded at the same time. The bandwidth graph at the bottom looks like it capped out for most of the time between 1 and 2.4 seconds.
Here is the same test with no bandwidth limit (basically a 35Mbps connection) but with the same 50ms latency: http://www.webpagetest.org/result/120718_QT_X13/
In that case you still have some penalty as TCP ramps up slow start. Here is what it looks like running flat-out: http://www.webpagetest.org/result/120718_78_X20/
It’s not a node consistency problem so much as it is a more realistic view of performance for end users.
Can you tell me why the bandwidth is capped out at 1.5 Mbps? Is this the normal cap for dsl on that node?
Thoughts?
In the advanced settings you can select the type of connectivity you would like to simulate. The native connectivity is a lot faster and then we use dymmynet to make it look like a realistic end-user connection.
1.5Mbps DSL is the default connectivity profile we use because it’s on the lower end of the residential broadband spectrum (75th percentile or so) - https://sites.google.com/a/webpagetest.org/docs/other-resources/2011-fcc-broadband-data
You’re welcome to use any profile you’d like but one of the biggest values that WebPagetest brings is the more-realistic end-user connectivity. If you test from datacenter nodes you usually have < 1ms RTT and 1GBps connectivity to CDN edge nodes which the CDNs sometimes colocate in the same datacenters (and even racks) as the testing company’s equipment. That results in unrealistic gains in performance when measuring CDN impact.
Thanks again!
Question, what do you mean by “running a flat-out”? And how does one determine where the bandwidth caps out at? Does the graph look to be capped out at 3.1 seconds also then?
http://www.webpagetest.org/result/120717_J9_1229dc79d0f7153a5d5e47c26275b47f/1/details/
Sorry, the “DSL” configurations cap bandwidth at 1.5Mbps. Measuring bandwidth on the client side can be a bit spiky so it’s not unusual to see peaks over that but the scale on the bandwidth chart says it is 0-2.5Kbps so just over half way would be the 1.5Mbps limit.
From 1s-2.4s the bandwidth line is hovering right around where I’d expect 1.5Mbps to be on the graph.
Running flat-out means 0 added latency and no bandwidth limit (custom profile with 0 for all of the settings).
From the image, how do you determine mbps being consumed?
The scale of the graph is in the label on the left (0-2.5Mbps). CPU utilization is always on a 0-100 scale.
Sorry I wasn’t clear. I see that it’s 0-2.5 mbps, but how by looking at the graph, how do I know what bandwidth is it using?
The X axis is the seconds in the page load, but the Y axis, how do I determine what points is .5, 1.0, 1.5, etc. mbps?
Hopefully that makes sense.
The top of the graph is 2.5Mbps and the graph is broken into 4 bands (grey/white alternating) so each chunk is roughly 631Kbps. 1/2 way up the graph would be 1.2Mbps, etc. Nothing scientific, mostly just eyeballing it.
Thanks for the clarification.