I just registered today, so I’m new to the newer features such as compare tests etc.
Ever since I registered, WPT seems to start 2 tests at once, one of which always has much poorer results:
As you can see, they always start at the same time (just a few seconds later).
Normally, I wouldn’t care, it can’t hurt to have more results. But the first of those tests always has a very low first byte score (F, while the other one has B) and in general the results differ quite a lot. This makes the results unsuable as I can’t tell which one is more accurate.
What do I make of this? How can I stop WPT from starting two tests at once?
I’m not sure if this happens ALL the time but I just wanted to know if one of those tests (probably the one with F as first byte time) is unreliable for some reason.
So, I’m not sure why or how it’s submitting two separate tests…could you try it in a separate Chrome profile with no extensions? I’m guessing it’s something like that tripping things up.
As for the TTFB, the variability appears to be due to Nginx caching.
When your TTFB looks good, you can see that it’s a cache hit.
When TTFB is slower, I consistently see a cache miss.
One test fires a little earlier than the other, and it looks like those tests where the requests hit the server first are the ones that get the miss. The requests for the subsequent runs all come a hair later and end up getting cache hits.
So long story short: they’re both valid, it’s just that one test ends up testing the TTFB with the proxy cache layer caching, and the other ends up testing without.
OK thanks, I’ll try that! In your opinion: Do I need to worry about those cache misses? Do you think they are a real issue or only caused by this duplicated firing of the tests? Thank you for your assistance!!
