AWS is reporting that all affected services are recovering, and they are now working on recovering impaired services.
This incident affected ION only intermittently affected with ~10% of its requests. Beyond EC2, most of the services affected by the degraded performance do not affect ION. We are resolving this incident as we are now confident in service availability.
[2:04 PM PST] We have executed a mitigation which is showing significant recovery in the US-EAST-1 Region. We are continuing to closely monitor the health of the network devices and we expect to continue to make progress towards full recovery. We still do not have an ETA for full recovery at this time.
We will continue monitoring for any issues until the issue is fully resolved.
Posted Dec 07, 2021 - 22:18 UTC
Identified
We have identified the issue and are monitoring information coming from AWS. Per our analysis, ION systems are more minimally affected compared to the broader affected AWS outage. The result is a 10% error rate in retrieving and saving data in ION. We will continue to monitor the issue from AWS and send updates here.
Posted Dec 07, 2021 - 20:46 UTC
Investigating
AWS' us-east-1 region is experiencing degraded performance (https://status.aws.amazon.com/). ION is experiencing degraded performance as a result. Certain transactions may fail but may succeed on retry. Please refresh for accessing data.
We will continue monitoring the situation and informing this channel
Posted Dec 07, 2021 - 17:43 UTC
This incident affected: API, Application and UI, GovCloud Platform and API, and GovCloud Application and UI.