Hopefully this will be a lot less impactful than the default browser switch but it looks like the 3G mobile profile may be a bit optimistic, particularly the latency.
The current numbers are 2Mbps down, 1Mbps up with 150ms of latency.
There has been a fair bit of data released since the profile was set (including one that I worked on with Matt Welsh from Google):
US Average was 344ms latency with 1.6Mbps down/768Kbps up.
Latencies more in the 100-300ms range and bandwidth all over the place from 500Kbps-4Mbps down and 500Kbps-1Mbps up.
Clearly the 150ms RTT on the current profile is on the fast side of things and probably too fast. The 2Mbps/1Mbps also appears to be a bit high.
I’m proposing a change to the settings for the “3G” profile to change to 1.6Mbps down, 768Kbps up and 300ms RTT (still with 0 packet loss to keep things nice and reproducible).
We have a speedtest.net node running on our network, and from the reporting from iOS, Android and Windows phones clients over 3G/HSPA we see an average of 2446/843 with 239ms latency.
I’m quite sure those test are made while trying to get the best signal as possible, since they test their (max) network speed with the app, so ‘daily’ usage will be even less optimal.
the update of the connectivity is a good idea since I found the previous settings to be rather unrealistic. On the private instances I manage for clients, I set it up to 2.5 Mbps/700 Kbps 200ms RTT, based on public studies made for my country (France). But since I found that optimistic, I also added a 25% packet loss.
I was wondering how exactly was simulated the packet loss ? Specifically, I would like to know if from one test to another the same HTTP requests will be affected or if it’s complete random for each test, adding a variability that is not wanted in synthetic tests
The packet loss is random which makes it less useful for the cases of synthetic perf testing (unless you are testing protocols or techniques for dealing with packet loss). It will add a bunch of variability depending on where the packets get lost (DNS and SYN packets are significantly more expensive than data packets).