Private Instance configuration

[starkers]

Hi guys, I’ve been trying to get a selt hosted system setup but I’m sorta struggling to figure out where I’ve gone wrong…

My layout is this:

a central web server, Centos x64, apache mods+php all working…
the webserver only allows http and https on egress.

lets call this http://foo.bar.com, I fixed permissions per documentation and have disabled selinux.
It has a public IP and I double checked its egress matches inbound.

its locations.ini looks like this:

[locations]
1=Ireland
2=Public_Dulles
default=Ireland

[Ireland]
1=IE
label=Test Location

[Public_Dulles]
1=WPT_Dulles_IE8
label=“WebPagetest.org - Dulles, VA”

[IE]
browser=IE 8
latency=0
label=“Ireland (amazon) - IE 8”
key=ZZZ

[WPT_Dulles_IE8]
browser=IE 8
label=“WebPagetest.org Dulles, VA - IE8”
relayServer=“http://www.webpagetest.org/”
relayKey=
relayLocation=Dulles_IE8

I have then setup one of the AMI instances with what I think is the the correct user data :
IE8 - ami-00b18074 : wpt-ireland/ie8-20110703.manifest.xml

user-data used on above ami:

wpt_server=foo.bar.com wpt_location=Ireland wpt_key=ZZZ

That being said there doesn’t apear to be much documentation on the firewall rules an ec2 instance should have, I’ve assumed it needs RDP and http+https inbound so thats all I has currently (and ping for now).

So at this point… does this look correct? I have checked the /install URL and it now reports that its not in communication with the remote:

From http://foo.bar.com/install:

WebPagetest Installation Check
PHP

PHP version at least 5.3: 5.3.3
GD Module Installed: yes
zip Module Installed: yes
zlib Module Installed: yes
curl Module Installed: yes
php.ini upload_max_filesize > 10MB: 20M
php.ini post_max_size > 10MB: 16M

Filesystem Permissions

{docroot}/tmp writable: yes
{docroot}/results writable: yes
{docroot}/work/jobs writable: yes
{docroot}/work/video writable: yes
{docroot}/logs writable: yes

Test Locations

No test agents are configured to render video


Ireland : Test Location
    IE : Ireland (amazon) - IE 8 - No Agents Connected
Public_Dulles : WebPagetest.org - Dulles, VA
    WPT_Dulles_IE8 : WebPagetest.org Dulles, VA - IE8 - No Agents Connected

What confuses me is that its saying its not connected to an agent but if I check the apache log I see:

54.228.XXX.YYY - - [27/Jan/2013:15:11:49 +0000] “GET /work/getwork.php?video=1&location=Ireland&key=ZZZ&ec2=i-bcdc27f6&pc=PAGEST-IE8&ver=342&freedisk=6.842&ie=8.0.6001.18702 HTTP/1”

XXX.YYY = the /16 on the amazon instance modified for privacy
ZZZ same key as in the other ZZZ’s above.

I’m hoping I’ve just done something really silly here as I’ve followed the instructions as best I could. Would love to write a propper howto once I figure out where I’ve gone wrong.

Obviously if you need more info just ask, Thanks very much!
-David

[pmeenan]

Try:

wpt_server=foo.bar.com wpt_location=IE wpt_key=ZZZ

For your user data. The agents need to connect to the leaf nodes of the location definitions.

As far as the firewall rules for the EC2 agents, they don’t need any holes punched at all. All of the requests are outbound (though RDP is nice if you want to remote in to take a look at them).

[starkers]

@pmeenan

Try:

wpt_server=foo.bar.com wpt_location=IE wpt_key=ZZZ

For your user data. The agents need to connect to the leaf nodes of the location definitions.

As far as the firewall rules for the EC2 agents, they don’t need any holes punched at all. All of the requests are outbound (though RDP is nice if you want to remote in to take a look at them).

Thanks yes that was just a silly misconfiguration on my behalf.

I’m experiecing strange black screenshots now

For example I just ran it 5 times against a very simple front-page.
Of that all had black screenshots except the 1st and 4th test… any ideas?

[pmeenan]

Black screenshots will occur if you RDP’d to the machine and then disconnected. That locks the desktop and screws up the screen shots. Just reboot the instance and they should start working again.

Though, that doesn’t explain how you got any screen shots at all unless you are still RDP connected and minimizing/restoring the RDP window.

[starkers]

@pmeenan

Black screenshots will occur if you RDP’d to the machine and then disconnected. That locks the desktop and screws up the screen shots. Just reboot the instance and they should start working again.

Though, that doesn’t explain how you got any screen shots at all unless you are still RDP connected and minimizing/restoring the RDP window.

Yes I was thinking the same thing, its hopefully booting up again now and we’ll see how it goes

[starkers]

@starkers

@pmeenan

Black screenshots will occur if you RDP’d to the machine and then disconnected. That locks the desktop and screws up the screen shots. Just reboot the instance and they should start working again.

Though, that doesn’t explain how you got any screen shots at all unless you are still RDP connected and minimizing/restoring the RDP window.

Yes I was thinking the same thing, its hopefully booting up again now and we’ll see how it goes

Well I’m glad to report that I have a working instance now, however after configuring and installing : IE9/Chrome/Firefox/Safari - ami-8b2c2fff : 314854558937/ie9-20121031 (Eu-West Ireland) I’m having very very bad results.

For example my second views on sites take longer than first views, sometimes over 30seconds (for a page that always loads under 5sec on a real PC)

[attachment=257]

I’ve automated tests and started graphing with cacti but the results do not reflect reality… so I did some checking and found that even though my config had asked for Chrome :

---

[locations]
1=all
default=Ireland

[all]
1=Ireland01
label=“ami instance in ireland - Testing”

[Ireland01]
browser=Chrome
label=“testing”
latency=0
key=XXYYZZ

I’m ran a test against http://www.whatismybrowser.com/ and the screenshot says I’m running IE 9

[attachment=258]

Could this explain why my results are so aweful? all pages I’ve compared against firebug for example are behaving completely differently (and slowly)

The fact that second views are slower (like clockwork) is really troubling any advice?

[pmeenan]

Couple of things.

1 - What size instances are you running? Medium work best, Small work OK, tiny would explain the really slow times you are seeing and should never be used.

2 - For Chrome/Firefox/Safari you need the location name to be Ireland01_wptdriver and then the browser= line can include all 3 (browser=Chrome,Firefox,Safari).

[starkers]

1. Ahh, yes I missed the drop down which sets the ami size when I created it!

2. That works a treat thanks