Linux EC2 AMI test agents

WebPageTest Private Instances
1

I’ve been using the Windows EC2 AMI test agents just fine for some time. Now I’m trying out the Linux version ( specifically ami-c4a968be in us-east-1, Virginia ). Oddly, I’m not seeing any attempt from the agent to connect to our private WPT server.

I confirmed that the instance can reach the WPT server. Pings come back fine, as well as a curl request to the WPT server. The WPT agent appears to be running ok:

python wptagent.py -vvvv --ec2 --xvfb --throttle --fps 30 --exit 60 --alive /tmp/wptagent

I’ve been poking around the image looking for agent logs some where that could give me a clue on what is happening ( or not happening ). Does it write logs out some where?

Any other suggestions on what to look at?

Ultimately I’m looking to have the WPT server manage spinning up / down the EC2 test agents, but one step at a time :slight_smile:

2

Possibly a difference in user data for the location string? The new agents only support wpt_loc and not the wpt_location field that had some baggage with it.

3

The user data string I have looks like this:

wpt_server=https://wpt.example.com/ wpt_loc=EC2-US-EAST-N-VIRGINIA

I used the real WPT server URL instead of example.com. I confirmed the EC2 instance has this same data in /var/lib/cloud/instance/user-data.txt.

Fortunately I noticed that in www/ec2/ec2.inc.php.
[hr]
Perhaps the agent doesn’t support a full URL value for wpt_server?

Hunting around, I came across - wptagent/webpagetest.py at master · WPO-Foundation/wptagent · GitHub

4

Confirmed, I replaced the wpt_server line with:

self.url = ‘{0}work/’.format(value)

I happened to include the trailing slash in the wpt_server value. After making that change I was able to get the agent to connect and run a test.

5

I’ve created a pull request for wptagent to support full URLs and HTTPS in the wpt_server option.

As noted in the request, my Python is still very basic, so I’m happy to have this done differently if there is a better option.

https://github.com/WPO-Foundation/wptagent/pull/63

6

PR merged. To be fair, the agent is my first real python project so it’s probably not a role model for python but it works :slight_smile:

The AMI’s auto-update the OS and agent code before launching the agents so they should be evergreen and automatically “just work”.

FWIW, I may not have documented it yet but there is also wpt_url as a param that takes the full url.

7

Thank you for getting that merged! My private WPT server has been spinning EC2 Linux agents up and down and I’ve confirmed that things are working as expected.

8

Awesome. Let me know if you bump into any issues or missing features. It should be pretty complete but there are a couple of settings that I’m still working on supporting (like fullSizeVideo) and some of the request manipulation options may not work in Firefox yet (adding headers, blocking requests, etc).