A cloud baseline is useful because it turns security expectations into something a team can evaluate repeatedly.
But a dashboard showing “compliant” raises another question for me:
Compliant across which accounts, Regions, resources, and rules—and based on what evidence?
A baseline is only as useful as the coverage behind it. A new account may not have the expected configuration. A resource type may not be recorded. A rule may not have enough information to evaluate. A centrally deployed control may also fail in a member account.
Those gaps matter because cloud environments change continuously. Teams create accounts, deploy services, add Regions, and make exceptions. I treat baseline validation as an ongoing operational process rather than a one-time deployment.
Define the expected state first
Before investigating drift, I want a clear answer to four questions:
What is in scope? Identify the accounts, Regions, and resource types that should be evaluated.
What should be deployed? Document the expected rules, settings, and version of the baseline.
Who owns it? Assign an owner for the central baseline and for remediation in each environment.
What exceptions are approved? Record the reason, scope, approver, expiration, and compensating controls.
Without an expected state, a dashboard can show what it knows but cannot tell me what is missing.
I also distinguish a policy decision from an implementation detail. For example, “security-relevant configuration changes must be visible” is the requirement. A particular AWS Config rule or automation is one way to support it.
Compare expected coverage with observed coverage
I would build an inventory of account–Region combinations that should be monitored, then compare it with the combinations reporting to the central view.
For each one, I would check:
Is the account active and expected to be in scope?
Is AWS Config recording in the Region?
Are the required resource types included?
Are the expected rules or conformance packs deployed?
Is the central aggregator collecting from that account and Region?
Are there deployment, permissions, or data-collection errors?
Is there a documented exception?
AWS says an aggregator collects configuration and compliance data across source accounts and Regions, but the data shown depends on recorder settings and the Regions selected for collection. The dashboard only displays rules that have compliance results. AWS Config
That is why I don’t interpret an empty result as proof that there is nothing to fix. It may also mean there is nothing available to evaluate.
Read the rule results behind the summary
A conformance pack can provide a useful summary, but it is not a substitute for the individual rule results.
AWS documents a subtle case: if some rules in a conformance pack are compliant and others have insufficient data, the pack can still show compliant. The pack shows INSUFFICIENT_DATA when all its rules cannot be evaluated for that reason. Rules can also finish evaluating at different times. AWS Config
Consider this illustrative example:
RuleResultRule ACompliantRule BInsufficient dataRule CCompliant
A green pack summary does not answer why Rule B lacks data. I would investigate its scope, required resource types, recording status, evaluation timing, and any errors before treating the baseline as fully evidenced.
This is also a useful reminder that different summary views answer different questions. I want to know whether I am looking at a resource, a rule, a pack, or an account-level rollup before drawing a conclusion.
Inspect recording and deployment health
AWS Config recording is a prerequisite for conformance packs. AWS also says accurate conformance-pack compliance reporting requires recording the AWS::Config::ResourceCompliance resource type. AWS Config
For an authorized, read-only spot check in a single account and Region, I might start with:
aws configservice describe-configuration-recorders \
--region <region>
aws configservice describe-configuration-recorder-status \
--region <region>
aws configservice describe-conformance-pack-status \
--region <region>Replace <region> with the Region being reviewed. These commands inspect configuration and status; they do not enable recording or repair a deployment. The caller needs appropriate read permissions, and the output covers the selected account and Region—not the whole organization.
I would compare the results with the expected inventory and investigate missing results or errors. For an organization-wide review, I would use an approved process to gather the same evidence across the relevant accounts and Regions, while recording access failures as gaps rather than silently skipping them.
Treat drift as a change to investigate
Drift is a difference between the approved expectation and the observed state. It does not automatically mean someone acted maliciously.
A difference might come from:
A newly created account that has not completed onboarding.
A failed deployment or missing permission.
A legitimate change that was not reflected in the baseline.
An approved, temporary exception.
A service or Region with different requirements.
An unauthorized or accidental configuration change.
I would document the observed difference, the expected state, when it was detected, the owner, and the next action. If the observed state creates immediate exposure, the response should match the risk; a routine review cycle is not a reason to delay escalation.
Automate carefully
Automation is valuable for comparing inventory, deployment status, and evaluation results. It can identify missing coverage more consistently than a manual spot check.
I would start with a read-only report and make failures visible. In particular, the report should distinguish:
Expected and reporting
Expected but not reporting
Reporting with noncompliant results
Reporting with insufficient data
Approved exception
Unable to determine because of an access or collection error
Only after the team trusts the inventory and understands the failure modes would I consider automated remediation. Some changes can interrupt applications or overwrite a legitimate exception. An automation role should have only the permissions it needs, and changes to its scope should be logged and reviewed.
This is Zero Trust applied to the control plane: verify the control, the scope, and the evidence. Defense in depth means combining configuration monitoring with change management, identity controls, logging, and accountable owners.
Measure the gaps, not just the green
A useful baseline report should show more than an overall compliance percentage. I would track:
Expected account–Region combinations versus those reporting.
Failed or incomplete baseline deployments.
Rules with insufficient data.
Recorder and aggregation errors.
New accounts awaiting onboarding.
Exceptions past their review date.
Time from drift detection to an owner and disposition.
The goal is not to make every dashboard green. It is to know where the baseline applies, where it does not, and what the team will do when reality changes.