This incident has been resolved, alerts have been re-enabled. Some customers might see gaps for the 07:45 - 08:15 UTC timeframe, these will be backfilled by tomorrow June 20th.
Posted Jun 19, 2018 - 05:37 EDT
Update
We're continuing to apply mitigations, next update in 30 minutes.
Posted Jun 19, 2018 - 05:04 EDT
Identified
The source of intake latency has been identified, we're applying mitigations.
Posted Jun 19, 2018 - 04:21 EDT
Investigating
We're actively investigating increased metric intake latencies. As an effect of latency, metrics on graphs may be delayed.
These delays may result in "no data" alert conditions for Metric Monitors, to avoid spurious alerts we've temporarily disabled these alert types.
It's important to note metrics are delayed but will be backfilled once all services are operational.
Posted Jun 19, 2018 - 04:01 EDT
This incident affected: Metrics and Infra Monitoring and Monitors.