AWS easy deployment first impressions

I spent the evening playing around with the AMI-based WPT deployment. Some questions and observations:

-Is port 80 wide open to the world pretty much just required? I spent some time trying to keep this locked down, but it looks as though the AWS test agent AMIs end up speaking over their public interfaces rather than the private ones, so the default security group allowing intra-security-group traffic doesn’t work.

-There’s a bug somewhere around here (webpagetest/ec2.inc.php at 5c054170fd015148da351bbcd77a049cc492d8fe · WPO-Foundation/webpagetest · GitHub) where it seems as though tags are attempted to be set before the instance is actually available. An ugly sleep(60) added before tag creation solved it, but I know the ruby aws-sdk provides a wait until ready function that may help here.

-The docs should probably mention to delete the SSH public key distributed with the server AMI. This, in /home/ubuntu/.ssh/authorized_keys could allow unauthorized access.

-since test agents aren’t launched with a key pair, the AWS console won’t let you reset the windows password.

I really love this new approach, but it’s definitely still a little rough and requires permissions that only make me comfortable running this on a dedicated AWS account that is only used for WPT infrastructure.

It does currently (though you could lock it down at the server level to only allow access to the /work interfaces). It shouldn’t be too hard to update the code and have it discover the private IP and have new instances come in through that (or even easier to have a setting with the server URL that should be passed for newly created agents).

Yeah, that was somewhat of a pain and resulted in orphaned instances. I removed the reliance on the tags for managing the instances so that shouldn’t be a problem but it’s disappointing that the tagging doesn’t “just work”. A cleaner solution might be to keep track of the instance ID’s and if they have been tagged and when we go through looking for the state of running instances to tag any that we know about that haven’t been tagged yet.

Thanks. Just added to the docs and next time I update the base image I’ll make sure to go in there and clean up before creating the image.

Shouldn’t be too hard to add a setting to let you specify a key pair that you’d like to use. Generally I usually treat the agents as completely throw-away black boxes and block all incoming connections but having options for tighter security wouldn’t be a bad thing.

Yep - I was mostly worried about getting it working first. Would you mind opening github issues for anything you find that you’d like improved?

I get nervous in general about the code being able to spawn and kill instances and maybe accidentally nuking other things in the accounts. One of the things I’ve been considering is providing an interface that cleanly provides the backlog for each location so users could set up AWS auto-scaling of agents instead of relying on my code to start/kill the instances. It would take a little more setup but it would give you a lot more control over it.