For the last two days I have trouble with WebPagetest.org (servers Falkenstein and also Dullas). It seems there is some kind of caching.
E.g. I am making changes on my site and test this changes in Firefox and Chrome via the network monitor. Everything seems fine (changes are visible).
But if I start a run on WebPagetest.org the old site - without the changes - was analyzed again. I have no explanation at the moment but it seems that WebPagetest will sometimes analyse old data.
Nope, it completely nukes the profile and does a fresh test every time. Do you have any caching proxies in front of your server (Varnish, Nginx, Cloudflare, etc) that may be caching the page?
Thanks Pat for the fast answer. I thought I could exclude the server (Nginx) for being responsible for that but it seems that this is the only source of error which is still left now. I have to do more investigation.