Hi Pat,
Are you aware that you can submit tests to WPT through the API and the online Portal and the test disappears?
Support told me last week that this is solved.
Looks like its back so something has regressed badly.
Some examples from the portal
Although the test is lost, the credit is taken from the account.
This is causing a major issue for our business, and I suspect for many others well.
Please sort it out
Phil
,
Sorry, I can help answer questions about the agent or browser and how they work (or questions about why sites are behaving like they are) because we use the agent for the HTTP archive and I still actively contribute to it but I haven’t been involved with webpagetest.org itself for ~2 years.
I don’t know how actively Catchpoint themselves participate in the forums. Support is probably the best way to get the dev team’s attention if you have something that’s not a general question.
Thank you for responding Pat.
I had not appreciated that you did not actually work on the project or at Catchpoint as I see you regularly comment so it was a poor assumption by me.
I have chased Support on the subject. This is a second occurrence of this problem which seems to me that they probably have coding management issues as it seems to be a clear regression.
It settled down again after a few hours of chasing yesterday so hopefully it will get fixed for good now.
However, are you able to surmise if this issue is related to the agent or server as the problem is sporadic and occurs in the API, online and in different regions.
Phil
,
Catchpoint has its own back-end test archive and management system (as well as test queuing and history). Which system the issue is in largely depends on if the test runs and then disappears or if the test disappears before even being run.
There are also logic for multi-server load balancing and CDN support so if there are issues with the load balancing config it could also explain it.
The UI uses the same entry point as the API so it isn’t surprising that it is not different between those and the locations mostly use the same infrastructure for the actual test management.
Sorry, from the outside, there are just a lot of moving parts where there could be an issue.
Thanks once again for your insight, I will just have to hope Catchpoint get this finally resolved so as it is very frustrating when the problem occurs as the test response with a successful submission and then goes straight to test not found. However, it manages to doc the credit for the test!
Regards
Phil
If it goes straight to “not found” and doesn’t go through the waiting UI or anything then it’s probably an issue with the CDN load balancing. Could be something else as well but I’m assuming you’ve given them the details for debugging.
+1 I am having the same issues. Had 5 consecutive tests that were a Load and Repeat Load started via API that also got the error of Test Not Found yet tests were still deducted from my Pro Account
Since you are a paid customer can you please email support@catchpoint.com and provide the customer information and the steps to reproduce it.
The Support Team informed me this week that the problem had been resolved by Development and it shouldn’t happen again. Therefore if you are having the problem you should report it and keep this thread update.
I have not had the problem for a couple of weeks now but I have been told before that this problem was fixed and all of a sudden it started up again, so I think they had a code regression and so it is possible it could occur again.
All I provided was screen shots from the test history together with a list of the failed test Ids. This is enough to identify everything about the issue.
Point taken. The two issues had the same symptom, but two different root causes. Our engineering team did add more tests to catch this in the future, and we are also monitoring it closer to catch them when it occurs.
Thanks for confirming this as 2 separate issues.
From here we had a reliance and expectation on the test working or at least enabling you to see why it didnt work through the API, so just disappearing was a bit surprising. We too have coded to detect it much better now we understand it can happen.
Is it possible to publish when changes occur so that we can see if any problem we detect aligns with any change in the server/agent ? Many online services enable you to track change and availability through online APIs and portals, this would be useful service management and availability feature
When we detect/discover a critical issue that impacts multiple clients we update the status page https://status.webpagetest.org/
We will discuss internally what we can do for when we do changes, like code releases, and also major bugs. In our Enterprise Platform we do notify clients in advance of code changes and also after the release we notify with what were the changes including any major bugs fixed.
Thank you for your feedback. Greatly appreciated!
Thank you for informing me of the status page - I was not aware of this and have now subscribed.
Unfortunately we have not grown large enough yet to get to the enterprise platform so anything that can be done to help advise of changes and feature releases etc would be greatly appreciated.