Webpagetest limitations

[zane]

Hi

I installed webpagetest about 6 months ago while doing some research into using it to make a website monitoring application. From my reading on the forums etc. there seemed to be some limitations that would prevent it from being ideal for my system. I just wanted to ask about two of the issues that I remember finding because I have just looked into webpagetest again and am seriously considering using it.

1. Concurrent tests. It seems that the model for webpagetest is based on a single queue of pages that need to be processed. Is there any way to process more than one page at a time with the current system? If not, is there a possible way around it (I am prepared to work on the feature myself). Is this a limitation in the packet capturing or the browser or something else?

2. The user needs to be logged in to get screenshots. I think I had problems with the screenshots or video when I was not directly logged in to the server (my memory is a little fuzzy). I tried hosting this on a VPN with remote desktop access. Does webpage test work correctly if there is a logged in user session without the user actually viewing the remote desktop? What is the recommended way to set up the server?

Also, I would like to use the scripting system to test a series of steps. I haven’t actually looked into this much yet but I will soon. I am just mentioning this here in case any problems/limitations with the scripting system come to mind.

Thank you for any help that anyone can give.

[pmeenan]

@zane

1. Concurrent tests. It seems that the model for webpagetest is based on a single queue of pages that need to be processed. Is there any way to process more than one page at a time with the current system? If not, is there a possible way around it (I am prepared to work on the feature myself). Is this a limitation in the packet capturing or the browser or something else?

Are you just looking for the ability to run more tests or do you need to schedule multiple to execute at EXACTLY the same time? Each work queue can have multiple agents pulling from the queue so you can run as many tests in parallel as you have agents (always could). The Dulles location for the public instance is running 16 IE8 agents for example. You just configure them all to pull from the same work queue.

@zane

2. The user needs to be logged in to get screenshots. I think I had problems with the screenshots or video when I was not directly logged in to the server (my memory is a little fuzzy). I tried hosting this on a VPN with remote desktop access. Does webpage test work correctly if there is a logged in user session without the user actually viewing the remote desktop? What is the recommended way to set up the server?

Yes, the user needs to be logged in. The easiest way to do it is to just configure Windows to automatically log in the user at boot time. If you RDP to a VPS, just reboot the system when you disconnect and it should work fine (That’s how the Delhi agent and the EC2 instances are running). Another option is to run VMWare ESXi on a physical box and run the test agents as VM’s (this is how Dulles is configured).

@zane

Also, I would like to use the scripting system to test a series of steps. I haven’t actually looked into this much yet but I will soon. I am just mentioning this here in case any problems/limitations with the scripting system come to mind.

The main limitation is that you can only record performance data for one of the steps in the script (well, to be honest, you can record them all but the UI gets confused). Better reporting for multi-step scrips has been on my to-do list for a while.

Thanks,

-Pat

[zane]

Hi Pat

Thank you for the detailed reply, that clears up most of my questions.

I just wanted to check something about your reply to my first question. Yes, I am just looking to be able to run more tests. You said that multiple agents can pull data from the same queue, but is it possible to have multiple agents running on the same machine? The server that I tried running it on has quite a high bandwidth, so it would be nice to have 5 different websites being tested at the same time on the same computer. That would mean that 5 different instances of internet explorer would have to pop up though.

With regards to the scripting, that’s not too much of a problem. I’d like to store the data that is returned in a database anyway, so I would probably need to use a different method to display it.

Thanks

[pmeenan]

@zane

I just wanted to check something about your reply to my first question. Yes, I am just looking to be able to run more tests. You said that multiple agents can pull data from the same queue, but is it possible to have multiple agents running on the same machine? The server that I tried running it on has quite a high bandwidth, so it would be nice to have 5 different websites being tested at the same time on the same computer. That would mean that 5 different instances of internet explorer would have to pop up though.

The main way we support running multiple instances on a single computer is to put them in VM’s (which won’t work on a VPS host). We used to support running multiple instances in parallel but a lot of things don’t work well in that model so we stopped supporting it:

• Start render, screen shots and video capture all require that the browser window be visible. With multiple browsers this means either small windows or an enormous desktop.

• The dummynet traffic shaping is difficult to do across multiple browser instances (to put it mildly). We had support for binding each browser to a different IP address and traffic shaping each of them independently but it wasn’t really worth the effort and still wasn’t a perfect solution.

VM’s just ended up being a lot cleaner.

[zane]

Ok great. Thanks for the explanation. It’s been helpful in getting me to understand how things work. I think I will get the system up and running on the VPS again and investigate it some more.