I know several users are running instances of the hosted version in EC2 and figured it would be a good idea for everyone to compare notes (and even AMI’s, etc). With any luck we could eventually get to a push-button deployment of private monitoring/WebPagetest in the cloud.
I’ve been running my WPT instance plus WPT Monitor for a couple months now on EC2. Originally I set it up on a fedora AMI, but have recently fired one up on CentOS as a micro instance. The CentOS micro instance seems to perform very well.
Here are the details of what I did. I believe most of these are required for the Monitor. I think WPT is less demanding.
I use RightScale to manage my EC2 resources.
I also have win2003 wpt agents in all EC2 regions.
If you try the following please provide feedback and I’ll tweak where needed.
Once we have this pinned down I will be able to create and share a pre configured AMI.
Micro instance
AMI: ami-2342a94a - RightImage_CentOS_5.4_i386_v5.5.9_EBS
yum install httpd
yum install php
yum install php-pear
yum install php-devel
yum install gcc
yum install zlib-devel
yum install curl-devel
pecl install pecl_http
yum install php-gd
yum install php-pdo
yum install subversion
echo “extension=http.so” > /etc/php.d/http.ini
cd
mkdir src
cd src
svn co http://code.webpagetest.org/svn/webpagetest/ webpagetest
svn co http://code.webpagetest.org/svn/wptmonitor/trunk/ wptmonitor
cp -R webpagetest/. /var/www/html/.
cp -R wptmonitor /var/www/html
// Edit settings
vi /var/www/html/settings/locations.ini
…
/etc/init.d/httpd start
What instance size are you using for your agents? The micro instances would be great if they were more specific about the CPU capabilities but I get the feeling that “short periodic bursts” are not sufficient for running IE and getting stable results but I’d love it if it does work well.
All my agents are win2003 spot micro instances. I only run 1 thread per instance and seem to be getting good results. With the spot instances my average cost per agent is about $15/month.
I haven’t verified the consistency of the results on micro instances yet. But they do seem to be sufficient for monitoring trends.
Attached is a waterfall from one of the micro instances compared to the same page pulled from webpagetest dulles instance. Notice the CPU utilization.
It would be nice to further confirm the numbers being produced by the micro instances.
The first image is from EC2 Micro Instance of windows 2003
The second image is from Dulles
btw, just to follow up in the thread, turns out Micro instances are not good for using for performance testing: http://www.webperformancetoday.com/2010/10/26/cpu-memory-issues-with-web-page-tests/
Small instances do look like they work well though. I am working on some public EC2 AMI’s that can be used for WebPagetest testers and configured through the instance user data so we should have sharable images for anyone to use soon. Just a few steps away from having a push-button monitoring system running in the cloud 
hi Pat,
does this mean that sometime soon I can easily install my own hosted instance of WPT, e.g. a private IE8 test node, and use WPT Monitor on my own server to do the monitoring?
I know little of EC2 … but how can I know for sure that IE8 test node will be in the Netherlands? Isn’t EC2 a global cloud … please educate me.
When you fire up an instance on amazon ec2 you specify the region. They currently have 4 regions.
us-west : California
us-east : Virginia
Asia Pacific: Singapore
eu-west : Ireland
Running a small spot instance of the wpt agent costs about $50/month. If you don’t need bandwidth shaping you can run 2 threads on the box with minimal impact.
Ah, so no way to have the tests run from Amsterdam, NL.
That is a showstopper for me.
Txs Tony.[hr]
The current Amsterdam IE7 test node is a normal WinXP consumer PC.
If I have an Apache server in NL and want to use that to setup the IE8 test node, how do I do that?
Use some kind of VM ware-ish software?
And, is that as good as a standalone consumer PC machine?
Right. You would need to set up your own wpt agent in Amsterdam. I’m hoping amazon will expand their coverage.
Is it possible to have my LAMP server become the IE8 node?
That would mean running Windows on Linux???
You would probably have to run vmware/virtualbox ( don’t know if wine would work ).
I’m not sure how well that would work out. If you try it be sure to let us know.
How frequent do you need the job to run?
Well, I am currently providing the public IE7 test node in The Netherlands for WPT.org. I want to offer the IE8 one as well.
Options:
- Buy a WinXP box … set it up …
- Somehow get it up & running on my LAMP server
option 2 seems like a trial & error and a hassle, but interesting to find out if it can work.
Pat > are you listening in? What are your thoughts?
I just wanted to throw out my thanks to everyone contributing – it’s amazing how straightforward it already is to get servers up and running. I’m running a combination of servers on Amazon EC2 and Rackspace CloudServer to get some better regional representation (though still working on having some ability to determine which data center the Rackspace instance comes up in).
I wanted to add an insight that may apply to some users that depend on distributed CDNs. One caveat I’ve encountered with Amazon is that the DNS servers used end up making requests that appear to served out of Seattle. The CDN I use has POPs (Points of Presence) in most major cities, but when using the Amazon US East nodes, they still end up using CNAMEs pointing at the CDN POPs located in Seattle rather than on the east coast. The Rackspace servers I run correctly resolve our CNAMEs to point at POPs near to the data center as I would expect.
The workaround that I’m using is to manually override the DNS settings on the EC2 instance, and have the first DNS server point at a public DNS (such as OpenDNS) and the second pointing at the originally assigned DNS (just in case there’s a need for it)
-andrew
I checked a couple of my servers and they seem to be operating as expected. When requesting a resource on akamai from my ireland ec2 instance i get an ip address that shows to be correct. I get the same for my CA and VA instances.
I’ll keep an eye out for this tho. I’ve had issues previously with some of my PCs having mis-configured DNS entries, so I know it can create some invalid numbers.
Regardless of which way you go you’ll need another Windows install (the licensing costs are a royal pain). You have a couple of choices:
1 - Set up another PC
2 - Run VMWare server on your LAMP box and install inside of there (would take some pretty significant testing to make sure the results are reliable)
3 - Set up a VMWare ESXi server and run both the IE7 and IE8 testers on VM’s (would require some testing but I have done a bunch of testing with ESXi and it is reliable).
4 - Find a cloud provider that has a presence in the Netherlands
FWIW, I have some public EC2 AMI’s configured that I’m doing some testing with. If it all looks good I’ll make them available for people to use. I have enhanced URLBlast to auto-configure itself based on user data so the AMI’s can come right up and start testing.
For the CDN issue, did you tracert to the IP’s or were you trusting WebPagetest’s IP lookup (it tends to not be all that reliable with CDN POP’s). Which CDN provider?
Amazon’s DNS servers should be anycast and available in each data center so it would largely be the CDN provider’s problem to geo-locate them correctly. Switching to Google DNS might be an option but there was a thread earlier on how Akamai has problems geo-locating through Google’s DNS.
Thanks,
-Pat
btw, I just checked my East-coast instances and the RTT to Akamai is 1ms and to MaxCDN is 3ms so it looks like they are working fine for CDN geo-location.
If you want to play with them, my AMI images are documented here: http://www.webperformancecentral.com/wiki/WebPagetest/EC2
I haven’t looked at CloudServer yet to see if I can do the same.
Thanks,
-Pat
Thanks for the response. I was using nslookup, actually, which should be accurate for the response. We’re using Internap’s CDN product, which should geo-locate correctly.
Here’s the output from nslookup, the first using the DNS provided via DHCP, and the second using OpenDNS, the third using Google DNS.
C:\Users\Administrator>nslookup www.cozi.com. 172.16.0.23
Server: ip-172-16-0-23.ec2.internal
Address: 172.16.0.23
Non-authoritative answer:
Name: cdce.sef004.internap.com
Address: 64.94.126.65
Aliases: www.cozi.com
cozicorp.http.internapcdn.net
http.sef004.internap.com
C:\Users\Administrator>nslookup www.cozi.com. 208.67.222.222
Server: resolver1.opendns.com
Address: 208.67.222.222
Non-authoritative answer:
Name: cdce.wdc007.internap.com
Address: 69.88.152.250
Aliases: www.cozi.com
cozicorp.http.internapcdn.net
http.wdc007.internap.com
C:\Users\Administrator>nslookup www.cozi.com. 8.8.8.8
Server: google-public-dns-a.google.com
Address: 8.8.8.8
Non-authoritative answer:
Name: cdce.wdc007.internap.com
Address: 69.88.152.250
Aliases: www.cozi.com
cozicorp.http.internapcdn.net
http.wdc007.internap.comThe instance is running in US-East, so the Washington, DC (WDC) lookup in the second and third case is what I would expect. The Seattle (SEF) lookup in the first, seems wrong.
I’ve tried to fire up the EC2 images mentioned, but I can’t seem to get anything out of them.
Q1. What, exactly, are they intented to do?
Q2. How are they meant to be controlled?
- is there an Administrator password?
- is there a web-interface on some port?
They are pre built and configured windows instances with the remote agent software already installed. When you fire up an instance you specify the userdata string which provides the configuration needed by the urlblast process.
A minimal example of the userdata string:
wpt_server=www.yourserver.com wpt_key=secretKey
If you fire up one instance in each region using the above userdata string then your locations.ini file would/could look like…
[locations]
1=EC2_Viginia
2=EC2_California
3=EC2_Ireland
4=EC2_Singapore
default=Dulles_VA
;
; Physical Locations
;
[EC2_Viginia]
1=ec2-us-east-IE7
2=ec2-us-east-IE8
label=“Amazon EC2 East ( Viriginia )”
[EC2_California]
1=ec2-us-west-IE7
2=ec2-us-west-IE8
label=“Amazon EC2 West ( California )”
[EC2_Ireland]
1=ec2-eu-west-IE7
2=ec2-eu-west-IE8
label=“Amazon EC2 EU ( Ireland )”
[EC2_Singapore]
1=ec2-ap-southeast-IE7
2=ec2-ap-southeast-IE8
label=“Amazon EC2 AP ( Singapore )”
;
; The individual configurations
;
[ec2-us-east-IE7]
browser=IE 7
latency=4
label=“Amazon EC2 East ( Virginia )”
localDir=/var/www/html/work/jobs/ec2-us-east-IE7
key=secretKey
[ec2-us-east-IE8]
browser=IE 8
latency=4
label=“Amazon EC2 East ( Virginia )”
localDir=/var/www/html/work/jobs/ec2-us-east-IE8
key=secretKey
[ec2-us-west-IE7]
browser=IE 7
latency=4
label=“Amazon EC2 West ( California )”
localDir=/var/www/html/work/jobs/ec2-us-west-IE7
key=secretKey
[ec2-us-west-IE8]
browser=IE 8
latency=4
label=“Amazon EC2 West ( California )”
localDir=/var/www/html/work/jobs/ec2-us-west-IE8
key=secretKey
[ec2-eu-west-IE7]
browser=IE 7
latency=4
label=“Amazon EC2 EU ( Ireland )”
localDir=/var/www/html/work/jobs/ec2-eu-west-IE7
key=secretKey
[ec2-eu-west-IE8]
browser=IE 8
latency=4
label=“Amazon EC2 EU ( Ireland )”
localDir=/var/www/html/work/jobs/ec2-eu-west-IE8
key=secretKey
[ec2-ap-southeast-IE7]
browser=IE 7
latency=4
label=“Amazon EC2 AP ( Singapore )”
localDir=/var/www/html/work/jobs/ec2-ap-southeast-IE7
key=secretKey
[ec2-ap-southeast-IE8]
browser=IE 8
latency=4
label=“Amazon EC2 AP ( Singapore )”
localDir=/var/www/html/work/jobs/ec2-ap-southeast-IE8
key=secretKey
You shouldn’t need an admin interface. The instances restart automatically periodically and can be manually restarted or terminated and re instantiated if there are any issues.