All monitor notifications are back to normal and the problem has been resolved. We don't expect any further impact at this time.
Posted Jan 06, 2021 - 17:31 EST
Monitoring
Mitigations are in place for monitor notifications and they are actively recovering. Some customers may continue to see some continued monitor notification delays as services recover. We will update again as the recovery continues.
Posted Jan 06, 2021 - 17:13 EST
Update
Work continues to mitigate the impact on monitor notifications. At this time, monitor notifications may still be delayed or skipped to avoid spurious alerts.
Posted Jan 06, 2021 - 16:22 EST
Identified
The issue has been identified and mitigations have been put into place. Dashboards and API requests should be fully functional now. Monitor notifications may still be delayed or skipped to prevent spurious alerts.
Posted Jan 06, 2021 - 15:51 EST
Investigating
We are seeing high error rates with metrics queries. This may result in dashboard errors, api request errors, and we are skipping monitor notifications to avoid creating spurious alerts. Investigation is underway and we will provide updates as soon as we learn more.
Posted Jan 06, 2021 - 15:44 EST
This incident affected: Metrics and Infra Monitoring, Monitors, and Web Application.