Any ideas what could be causing this? Seems very consistent that it’s about 10sec and every 20 requests.
This one from Gloucester UK, using IE7 (site is in UK)
http://www.webpagetest.org/result/120607_R6_PQR/
And this one from Dulles using Firefox. I’ve cut the file sizes down since this, but the pattern remains.
http://www.webpagetest.org/result/120607_MS_G5Y/
Thanks,
Ian.
That’s not going to be file sizes. That is either packet loss on the network or a web server that can’t handle any more connections and it’s taking a few retries before the connection succeeds. I’d reach out to your hosting provider to have them take a look. It’s not something that is going to be in your control and is unrelated to file sizes.
While they are looking you can also ask them why they don’t support persistent connections and if they can turn it on (my guess is that they don’t support it because the server is overloaded and that’s also the cause of the long connection times).
Odds are they will push back and say it’s not their problem so if you can run tests from a few locations that reproduce the problem and enable tcpdump capture with the tests it should give you more ammunition.
I was convinced it couldn’t be anything I was doing, thanks for confirming that. I’ve been on to the hosting company 3 times now and, as you say, they just insist their servers are OK and it’s not their problem. They won’t turn persistent connections on and they won’t look any further. So I guess I have no choice but to move, although I’d really like to understand the cause first. I ran another test:
http://www.webpagetest.org/result/120612_59_G3M/
with tcpdump turned on, but couldn’t see what difference it made? What should I be looking for?
Thanks again for your help.
Ian.
Sorry, you won’t see a difference in the result with tcpdump enabled but you will get a link to download the tcpdump (packet capture) of the test. The tcpdump will show the packet loss very clearly with the syn retransmits.
Since it’s very reproducible for your site and doesn’t happen to others from the same test agent that means the problem is either with your hosting provider’s network or server and the tcpdump should be enough evidence (along with clean tests) for a reasonable engineer to dig into it on their side.
I looked at the packet capture and can clearly see the SYN retransmits (connection setup) that cause the really long orange lines so packets are either getting dropped or the server can’t respond.
Given the lack of support for keep-alive, it’s quite possible that the hosting provider just doesn’t care which would be rather unfortunate. If they have public forums for discussing issues I’d recommend posting there as well so that other users are aware and it might drive them to fix it.
The end, at last! After 3 weeks and many emails from ‘support’ denying there was a problem and suggesting my web pages needed optimising and I should ‘consult a web developer’ (had to hold my breath and count to ten after that!), it’s finally fixed.
In the end I assembled enough evidence, by copying my code to another site, where it worked perfectly, copied another site to my domain where it showed the same problem, and used reverse IP lookup to find other websites on the same server, most of which also had the same problem! Finally but I think most significantly I quoted your post above about tcpdump and the SYN retransmits. This seems to have brought it to the attention of someone who knew what they were doing and the problem was fixed in a few hours.
So thanks again for your help, a very useful site and my understanding has also increased a lot!
Regards,
Ian.
That’s great news. Sucks you had to do so much work just to get them to look at it though. Did they fix the persistent connections while they were at it or just the packet loss?
Just the packet loss. But the site is very responsive now, so I’m happy to leave it at that.
Regards,
Ian.