I conducted a test on www.cnn.com on May 19, 2011 at 13:30 CST and compared the data in the “Request Details” table (via the WPT GUI) with the data in the “requests.csv” file for the same test.
Observations:
- There are only 172 rows of data in the former (GUI table), but 176 rows of data corresponding to the “Cleared Cache-Run” case in the latter (CSV file).
- Apart from the “Time to First Byte” column, I was not able to match the data in many of the other columns.
Am I misreading/misinterpreting something or is the data from these 2 sources in fact inconsistent?
Assuming the 2 sources are NOT compatible, what would be the best way to download the former (i.e. GUI table) via scripts? (Hoping to avoid having to wade through HTML code to extract the data…)
Thanks,
Anshul
Well, since the CSV is actually used to draw the UI I don’t think it’s a mismatch problem, possibly just a processing difference.
The first thing you need to do is only look at “Transaction Type” == 3 - those are the requests (type 0 is the summary page data and 8 shouldn’t be in the data but looks like it’s sneaking through). That will get you to the 172 rows.
After that there is a bunch of adjusting and calculating that is done to get the individual times (mostly because some of the component times are included in the “load time”).
The “DNS Lookup” in the GUI table is the same as the “DNS Time” column
The “Initial Connection” in the GUI table is the same as the “Connect Time” column
The TTFB you already have matching
The “Content Download” in the GUI is “Time to Load” - “Time to First Byte”
The tricky one is Offset. The “Start Time” in the CSV is the offset of the request, not including the DNS, socket connect or SSL. To get the offset you need to subtract the DNS, Connect and SSL times from “Start Time”
Indeed, based on some random checks, the data from the CSV does match up to the GUI table as per your instructions. Relieved!
Thanks - I would have never figured out those mappings myself…
Regards,
Anshul[hr]
Just to get a quick confirmation from you:
Although there is no equivalent GUI table for the Repeat View on WPT (right?), if I look at the CSV file, I can follow your instructions (i.e. look at only rows with “Transaction Type” == 3, etc, etc) for the rows corresponding to “Cached-Run” in column C, to derive similar data (Start Offset, TTFB, etc) for the Repeat View, right?
Thanks,
Anshul
If you click on a repeat view waterfall you should be taken to the equivalent page that has a repeat view data table. You can either use the name or the explicit “cached” column to distinguish.
How can I determine the “Start Render” and “Document Complete” times (green and blue lines respectively in the GUI’s Waterfall View) from the CSV files?
Thanks,
Anshul
Those are in the page data csv file (should have columns for both of them).