All older events have been reindexed and all systems are operational.
Posted May 28, 2017 - 10:20 EDT
Update
We are continuing to reindex some events from May 26th and 27th. All event alerts on events received after 1am UTC on May 27th are operational. All other types of alerts are operational. Earlier event alerts are coming back online as data is reindexed.
Posted May 27, 2017 - 20:40 EDT
Identified
We are reindexing some of today’s events once more. All event alerts on events received after 6pm UTC are operational. All other types of alerts are operational. Earlier event alerts are coming back online as data is reindexed.
Posted May 27, 2017 - 17:50 EDT
Investigating
We have identified new issues relating to the reindexing of events and are working to put a fix in place. The events page might also show 5xx errors which we are working on, however, other pages will continue to operate as expected, e.g. https://app.datadoghq.com/dash/list
Posted May 27, 2017 - 12:06 EDT
Update
We’re still reindexing some of today’s events. All event alerts on events received after 1am UTC are operational. All other types of alerts are operational.
Posted May 26, 2017 - 22:33 EDT
Update
Short term event alerts enabled. Longer term event alerts will follow shortly. All other alerting types are unaffected.
Posted May 26, 2017 - 21:22 EDT
Update
Mitigations are still being applied to the Events system, we’ll update with new development as soon as possible.
Posted May 26, 2017 - 20:57 EDT
Update
We are applying a mitigation to our Events system, next update in 30 minutes.
Posted May 26, 2017 - 20:26 EDT
Update
Event search is returning inconsistent results, we're investigating. Event alerts have been disabled to prevent spurious notifications.
Posted May 26, 2017 - 19:48 EDT
Identified
Event search is returning inconsistent results, we're investigating. Event alerts have been disabled to prevent spurious notifications.
Posted May 26, 2017 - 19:47 EDT
Investigating
Event search is returning incorrect results, we're investigating.