A bit confused about chrome results

Hi there,

I just ran a quick test of a page on our website in chrome.

http://www.webpagetest.org/video/compare.php?tests=120315_VV_3KN46-r:1-c:0

compared to the time line in IE it is pretty horrible: IE below

http://www.webpagetest.org/video/compare.php?tests=120315_4V_3KN9C-r%3A1-c%3A0&thumbSize=200&ival=100&end=all

I looks like chrome is not requesting any associated content (css or js) until was after it gets the whole page.

When I run the same test locally in chrome I can see chrome downloading resources in a waterfall more similar to IE, real world stat in fact suggest that chrome is the fastest to render this page.

Am I doing something wrong? Are there any issues around chrome captures (with video enabled)

Sam

There aren’t any known issues but I’ll look closer to see if I can figure out what is going on. It doesn’t look like it is related to the video capture because I see it doing the same thing in non-video tests: http://www.webpagetest.org/result/120316_89_2fff53a4333bf9319f9f29e4273020a0/

It’s possible that some of the APIs our extension connect to are changing the behavior or some of the command-line flags we use to run it. I just opened a critical bug to track it but it may take a few days to nail: Google Code Archive - Long-term storage for Google Code Project Hosting.

Interestingly, in chrome the onload event looks like it was pushed out beyond when a lot of the async content loaded (where in IE it fired before the async content).

Thanks,

-Pat

Just to keep things interesting, I can’t reproduce the issue in my dev VM’s - ARGH!

http://www.webpagetest.org/result/120319_13_5831156a347bec6361edeef6ceb494c5/

Good news is that that means it’s probably not related to the actual instrumentation code - but I need to figure out environmentally what is different on those machines to trigger it.

ok, some progress. It looks like Chrome isn’t actually waiting for the full base HTML to download, it is just a lot slower to spin up the additional requests. For a site where the base page takes longer to download you can see the overlap that you would expect:

[attachment=176]

IE (on the test machines anyway) is a lot faster to get started on the subsequent requests:

[attachment=177]

Historically when I saw a similar problem it was because chrome would start a new renderer process right there which caused it to be slower. It was a problem where they were not re-using the about:blank renderer by default so I switched chrome to “process per tab” mode to get it to re-use the same renderer going from about:blank to the measured page. That doesn’t appear to be the problem right now but I’ll see if I can track it down further.

The good news is that the behavior is actually accurate when navigating from about:blank to the tested page in a new browser session - I just don’t know why it is happening yet.

Patrick,

I had Jarrod run a test from his place that has 100ms latency and he got this:

In his test Chrome is able to grab the CSS pretty early.

I may be totally off, by I wonder if you could navigate to a static html page on the c drive, in web page test, prior to running the clean cache test. That way you would possibly spin up the bits that chrome needs in the tab prior to measuring.

I guess there are 2 unprimed cache cases:

  1. first page you navigate to in a tab
  2. subsequent pages navigated in the tab

Since Google is everyone’s home page, I would argue that 2 is more common. Our real world stats suggest chrome is the fastest to first render (just behind IE10 at the moment).

Yeah, I’m not giving up on it yet. Here is what it looks like navigating to www.gogle.com first and then going to the page: http://www.webpagetest.org/result/120319_EY_3760382567176743033867b1f5eeae76/ (the penalty is gone).

I’m trying to figure out what bits in Chrome aren’t initialized and see what it takes to get them warmed up (assuming it is “fair” and it wouldn’t be cheating - like loading the recent apps page).

I can also capture the timeline data from Chrome and record it with the test data to compare to WPT’s view of the world to see how they line up (the timeline is from the perspective of the renderer and there are some IPC jumps that could show up differently). In normal operation Chrome starts a new renderer process for each domain so things get pretty muddy looking beyond just the site performance itself (not to mention all of the predictive fetching and lookups that get disabled because I’m using a clean user profile).

Anyway, I’ll post here as I try a bunch of different things.

Out of curiosity, how are you measuring first paint in the field? It’s not an event that the browsers expose to script - I’m assuming you have a proxy of some kind? It’s also worth noting that Chrome had a bug in it’s navigation timing implementation where the start time was not including the renderer startup times and was being recorded later than it really should have been (not sure if that impacts your measurements at all).

Thanks,

-Pat

btw, did Jarrod have 100ms latency on the last mile or round trip to stack overflow? The reason I ask is that the CDN request for the css looks like it was a lot faster.

Jarrod has 100ms to SO, but the CDN (netDNA/MaxCDN) is closer to him AND they uses fastsoft (http://www.fastsoft.com/home/) which totally cheats on TCP slow start, so it looks pretty fast.

In the field we place a JavaScript probe just before the body

// in top

// before body

We report this + all window.performance probes to a central server for 10% of our requests.

btw … ie9/10 do report msFirstPaint in navigationTimings which can be handy

Much better now: http://www.webpagetest.org/result/120320_A0_3NE2T/

Current theory is that Chrome is setting up the various caches on first access (when it notices that it’s not there because I’m testing with a clean profile).

Instead of starting from about:blank I made a change so that the browsers start from a blank HTML page that is served from localhost on the test agents (they already have an embedded web server for talking between the extension and the c++ code so it was an easy change).

I feel pretty good about that as not being “cheating” since I’m doing it for all of the browsers and users will hardly ever have a completely clean user profile.

Thanks again for pointing it out.

-Pat

Thanks heaps Pat! Very happy you figured this one out