I just migrated one of my agents from URLBlast do WPTDriver.
Unfortunately some of the tests started to fail. This applies only to tested sites that require NTLM authentication. The problem is that in waterfall I now can see all the responses of the server to the browser that require to provide credentials:
After that browser sends some hashes to the server and page loads as expected. All subsequent connections to the server are authenticated.
This is great because UrlBlaster does not separate this behavior from reported http requests. And it is impossible to see how fast it is done and TTFB is reported incorrectly.
The only problem is that after that (in my opinion normal behavior of the server and browser) the test has error status 401.
Is this big deal to correct this? My automated tools that use WPT Server have problem with reporting statuses of tests. They are not able to distinguish described case from real problem with authentication.
I investigated this issue deeper and noticed that for described case when result for run is other than “0” average is not calculated. Only median. Tools that rely on average of runs have problem in case of “401” responses from the server.
There is another problem with tests of pages that require authentication by sending 401 responses. Repeat view test is not performed:
2014/04/18 11:54:02 - Test Created
2014/04/18 11:54:06 - Starting test (initiated by tester PLKRK-L-R9A1FAP-10.3.52.208)
2014/04/18 11:54:37 - Test Run Complete. Run: 1, Cached: 0, Done: , Tester: xxx-10.3.52.208
2014/04/18 11:54:37 - Test Run Complete. Run: 1, Cached: 1, Done: , Tester: xxx-10.3.52.208, Test Run Error: "Skipped repeat view, first view failed: Test run failed with result code 401"
2014/04/18 11:54:37 - Test Run Complete. Run: 1, Cached: 1, Done: 1, Tester: xxx-10.3.52.208, Test Run Error: "Skipped repeat view, first view failed: Test run failed with result code 401"
2014/04/18 11:54:37 - Test Complete
2014/04/18 11:54:38 - Test post-processing complete
Any updates on this bug? We’re looking at implementing a private instance to test our internal web sites, most of which use NTLM auth. We noticed the same problems.
It’s valuable for us to capture the 401s but as mentioned before we don’t want the test to report as an failure.
Is it just the 401’s or is there also a functional problem getting to the page? It should be easy enough for me to treat 401’s more like redirects which will make them non-errors and report a correct TTFB.
In my case it’s just the 401s, the first test completes as you would expect (captures the entire waterfall) just reports as an error and skips the repeat test (plus the other issues noted by OP). Treating them more like redirects sounds like an appropriate solution.
Now I have the same issue with “407” proxy authentication requests:
When getting “HTTP/1.1 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )” response agent reports “First View (Error: 407)”
Any chance you can email me a screen shot? It sounds like a different failure where the auth actually failed rather than the auth succeeding but being reported as a test error.
Any chance you can email me the result folder for that test as a zip file? If I can reproduce it I might be able to get a fix in the 2.15 release that I’m preparing right now.
Sorry, just realized the original fix was for 401’s and now you are seeing it for 407’s. Do you ming opening a github issue so I don’t forget? I’ll try and get it fixed tomorrow (though I just cut 2.15 so it won’t get into an official release for a while).