I have a CloudwatchAgent emitting High-Res metrics from my ASG. The config looks like this:
// AMAZON-CLOUDWATCH-AGENT.JSON
{
"agent": {
"metrics_collection_interval": 10,
"run_as_user": "cwagent",
"debug": true
},
"metrics": {
"aggregation_dimensions": [
[
"AutoScalingGroupName"
]
],
"append_dimensions": {
"AutoScalingGroupName": "${aws:AutoScalingGroupName}"
},
"force_flush_interval": 10,
"metrics_collected": {
"cpu": {
"drop_original_metrics": [
"cpu_usage_active"
],
"measurement": [
{
"name": "cpu_usage_active",
"rename": "CPUUtilization",
"unit": "Percent"
}
],
"metrics_collection_interval": 10,
"resources": [
"*"
],
"totalcpu": true
},
"mem": {
"measurement": [
{
"name": "mem_used_percent",
"rename": "MemoryUtilization",
"unit": "Percent"
}
],
"metrics_collection_interval": 10
}
},
"namespace": "ASG_SLB_HighRes"
}
}
And a corresponding Alarm with this config:
{
"MetricAlarms": [
{
"AlarmName": "Some-Name",
"AlarmArn": "Some-ARN",
"AlarmConfigurationUpdatedTimestamp": "Some-Timestamp",
"ActionsEnabled": true,
"OKActions": [],
"AlarmActions": [],
"InsufficientDataActions": [],
"StateValue": "OK",
"StateReason": "Threshold Crossed: 1 datapoint...",
"StateReasonData": "",
"StateUpdateTimestamp": "Some-Timestamp",
"MetricName": "CPUUtilization",
"Namespace": "Some-Namespace",
"Statistic": "Average",
"Dimensions": [
{
"Name": "AutoscalingGroupName",
"Value": "some-asg"
}
],
"Period": 10,
"EvaluationPeriods": 1,
"Threshold": 50.0,
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "notBreaching",
"StateTransitionedTimestamp": "Some-Timestamp"
}
],
"CompositeAlarms": []
}
Period: value is already the minimum of 1 as per AWS Doc
EvaluationPeriods: also set to 1 <- minimum possible
DatapointsToAlarm: Have not set it, since it just then defaults to EvaluationPeriods
However, the Cloudwatch Alarm goes to ALARM State about 25-26 seconds after the breaching data point. I have noticed that there is always a difference of about ~25 seconds in the queryDate and startDate in the StateReasonData of alarm. It is always around 25-26 seconds!
for example:
This is the StateReasonData from the Alarm History:
"StateReasonData":{
"version": "1.0",
"queryDate": "2026-07-10T20:10:05.419+0000"
"startDate": "2026-07-10T20:09:40.000+0000",
"statistic": "Average",
"period": 10,
"recentDatapoints": [
50.000
],
"threshold": 50,
"evaluatedDatapoints":[
{
"timestamp": "2026-07-10T20:09:40.000+0000",
"sampleCount": 9,
"value": 50.000
}
]
}
Any actions that I have attached to this alarm, hence, trigger after this lag of ~25 seconds. But this defeats the entire purpose of having high-res metrics! I have searched across the internet, AWS Docs and LLMs -> and I can not explain this delay.
What is this delay? Is this expected? Where can I find more about this? Or if there is further room for optimization in the configs above?