I have a private instance which is running benchmarks overnight - some from eu-west-1 and some from us-east-1. Everything working just fine up until the 24th March. Then from 25th March tests run from us-east-1 on IE8 are failing to run. Manual tests initiated from the UI also fail for IE8 in us-east-1. Same tests run from eu-west-1 run just fine.
When a test is initiated, I can see the 'Agent instance being created (ami-406f9429), but the tests never run (stuck in the queue). Terminating the agent, clearing the WPT queue and re-starting WPT make no difference - same result.
The agent instance appears to be always stalling (EC2 console reports 1/2 status checks failed - “Instance reachability check failed”).
Has there been some recent change to the Agent AMI or code that could cause this issue (since we’ve not changed config in this timescale)?
I’m now having the same problem in eu-west-1 - IE8 test agents (ami-00b18074) just hang (fail the EC2 reachability test), so IE8 tests are never run.
Anyone have any ideas?
Still failing. From the logs:
www/ec2/log/ec2.log.20150512:09:40:07 - Instance i-17937ebd started: m1.medium ami ami-00b18074 in eu-west-1 for eu-west-1_IE8 with user data: wpt_server=myprivateinstance.net wpt_location=eu-west-1_IE8 wpt_key=XXXX
www/ec2/log/ec2.log.20150512:10:35:07 - Terminated instance i-17937ebd in eu-west-1
So the ec2 log is saying the instance is terminated, but its clearly not (still listed as running in the AWS console).
This IE8 instance is being created every hour by WPT. My settings:
; EC2 Instances
; Should we automatically delete any EBS volumes marked as “available”?
; This can be used to prevent orphaned volumes but only if the account
; doesn’t expect to keep offline EBS volumes.
;Settings from user data