Delayed Processing for a Subset of Metrics
Incident Report for Datadog US1
Resolved
This incident has been resolved.
Posted Aug 02, 2023 - 01:45 EDT
Update
Distribution metrics lag has been resolved. As a result of the issue, some users may see gaps for distribution metrics on graphs. Monitors evaluation is now re-enabled for monitors based on this data.
Posted Aug 01, 2023 - 23:08 EDT
Update
We continue to make progress on processing the lag on distribution metrics and are monitoring any further issues.
Posted Aug 01, 2023 - 20:43 EDT
Update
We continue to process the lag on distribution metrics
Posted Aug 01, 2023 - 18:57 EDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Aug 01, 2023 - 17:42 EDT
Identified
The issue has been identified and a fix is being implemented.
Posted Aug 01, 2023 - 17:37 EDT
Update
We are continuing to investigate this issue.
Posted Aug 01, 2023 - 17:37 EDT
Update
We are continuing to investigate this issue.
Posted Aug 01, 2023 - 17:07 EDT
Investigating
We are investigating increased latency processing Processing for Distribution Metrics coming from the agent.
As a result of this issue, some users may see delays or gaps for a subset of their metrics on graphs.
RUM/Synthetics/Logs/Processes measures that depend on distribution metrics will also see delays.
To prevent spurious alerts, we have temporarily disabled monitors based on this data.
Posted Aug 01, 2023 - 17:05 EDT
This incident affected: Log Management, Metrics and Infra Monitoring, Monitors, RUM, and Synthetics.