in the last weeks I’ve setup a new server that should replace our very outdated wpt/osm instance. This new server contains an instance of webpagetest (newest private instance, running on Debian Stretch) as well as two agents, capsuled in Windows 7 kvms. On top is a docker instance of open speed monitor that controls and receives results from the wpt.
Basically everything runs fine and while monitoring the whole system(s) through nagios I can see that the load is very well balanced, means there is no system (neither Linux host nor kvms) that gets an overload or out of ressources.
However, there is one thing I can’t explain to myself: I noticed that (even though the new kvms perform significantly better in terms of reaction time than the very old wpt system where the agents ran on the same windows system where the private instance runs) it takes about 30 minutes longer for an agent to complete all the tests (about 40) than on the old system. When looking at the agents I saw that this is the result of a comparably long phase of the wpt-driver saying “Processing Result” after a browser is finished. But in this time, the agent idles or at least it seems so - no noticably cpu or memory consumption, no heavy network… It stays there for about 40 seconds to a few minutes before starting the next job.
So my question is: What exactly is happening at this stage? Communication with the private instance? On which site does the work happen here? I weren’t able do find anything in the docs or any option to eneble more logging.
I hope somebody can help me out here.