Robustness of EC2 Autoscaling

WebPageTest Private Instances
1

Hey Patrick -

I have been frustrated with some of the performance of agents in specific locations on webpagetest.org so am looking to create a server and agents of my own. Was ecstatic to hear that you have a server image on EC2 that is set to autoscale depending on usage!

However, looking through the code, I’m wondering how ‘public-ready’ is this? Here are the things that caught my attention:

  1. Reference to this url (http://169.254.169.254/latest/user-data) in www/cli/ec2init.php
  2. Summary emails going directly to pmeenan@webpagetest.org in www/cli/scrub-ec2.php
  3. Am I correct that it’s set to start up m1.small instance types? Those are pretty old and from my testing under spec of what’s needed to have reliable results.

Not trying to critique your code (I love what you’ve done with WPT), just curious if this really can be done in 5 minutes according to these instructions or if I should expect some headaches and debugging?

Thanks a ton!

2

FWIW, none of the cli code is used in the newer auto-scaling code.

The EC2 scaling code is here: webpagetest/ec2.inc.php at master · WPO-Foundation/webpagetest · GitHub

The shutdown check for instances are triggered every 5 minutes here: webpagetest/5min.php at master · WPO-Foundation/webpagetest · GitHub (no need to hook it up to actual cron, it is triggered automatically as agents poll for work).

The code in cli/ is from a few years ago and is more of a sample if people wanted to use it as a base for EC2 instance scaling and it was used on the public instance for some testing.

The instance size is m2 medium by default I believe but it can be overridden in the locations: webpagetest/ec2_locations.ini at master · WPO-Foundation/webpagetest · GitHub