Hi,
I wanted to run Lighthouse tests on our private WebPagetest setup in EC2. Right now we use Windows agents from ami-87f2ebeb on m4.large instances. However, I’ve read that Lighthouse works only on Linux agents.
So, I added Linux ami-e62e8189 as a separate location. The instances started without any problems, and tests work, but there are two problems that prevent this setup from being usable:
-
The tests are super slow. We run around 160 tests scheduled every hour, and 5 Linux AMIs finish them 3 times slower than 3 Windows instances. What’s more, the speed is the same on t2.medium and m4.large instances.
-
The mobile Chrome tests with “Emulate Mobile Browser” selected appear to just be changing the user agent, as I get mobile pages, but with desktop screen resolution. It appears that the resolution and dpi settings of mobile browsers are not applied in Linux AMIs. We’re using “Iphone6”.
Is there something wrong I’m doing? Is there anything I could debug to see where the problems lie? How do I log in to those Linux Agents? The Windows ones have user and password, but I can’t seem to figure how to get to the Linux ones.
[attachment=740]
Attached file shows the queue for Windows and Linux agents. Somewhere in the middle there I’ve switched from t2.mediums to m4.larges and it didn’t make a difference.
Thanks for the help!
Stay away from t2 instances at all costs. It’s not the root cause here but the variable performance is not suitable for performance testing. c4.large is pretty much the sweet spot for agents though in all cases it shouldn’t be that much slower.
As far as logging into the VM’s, when you start the instances you provide a ssh key to use and you use that to ssh in as the user “ubuntu”. If you are starting them through the API then you need to make sure you are passing the key.
Possibly more useful for debugging the performance, if you pass “debug=1” as a test option, the linux agent will capture a full debug log with timestamps. A download link for the debug logs will be to the left of the waterfall thumbnails where the timeline downloads are.
As for mobile emulation, something else is going on. Are you using the device profiles in mobile_devices.ini (in which case it is mobile=1&mobileDevice=iPhone6) or are you using script commands to explicitly set the viewport, scale factor, etc? Of the public locations, the Mumbai location is using the same AMI (lots of others are using the linux agent) and it is emulating mobile file: WebPageTest - Running web page performance and optimization tests...
Just a sanity check, were all 5 agents pulling work? Just wondering if something went wrong and only 2 or 3 were running.
Generally they should be slightly slower in processing but nowhere near that much slower. We migrated the HTTP Archive over from the Windows agents to Linux and maintained the same test thruput as before so something isn’t working right.
I’m using “mobile=1&mobileDevice=iPhone6” when creating URLs.
Yup, all 5 agents were pulling work and doing tests at the same rough speed.
I’m going to log in to the agents and do more debugging then. I’ll update the topic when I have new findings.
thanks for the help!