Automate Amazon Web Services Config reporting for noncompliant resources that have been non-compliant for a period of time

Amazon Web Services Config evaluates the configuration settings of your Amazon Web Services resources. You do this by creating Amazon Web Services Config rules , which represent your ideal configuration settings. Amazon Web Services Config provides customizable, predefined rules called Amazon Web Services Managed Rules to help you get started. While Amazon Web Services Config continuously tracks the configuration changes that occur among your resources, it checks whether these changes violate any of the conditions in your rules. If a resource violates a rule, Amazon Web Services Config flags the resource and the rule as noncompliant.

In this post, we provide a solution that can be used to automate Amazon Web Services Config reporting for resources that have been noncompliant for a period of time. You can choose the day and time for the report generation and get a list of resources that have been noncompliant for more than 30 days. Using this solution, you can avoid notifications from transient changes to your resources.

How Amazon Web Services Config Works

Amazon Web Services Config is a service that enables you to assess, audit and evaluate the configurations of your Amazon Web Services resources. Amazon Web Services Config tracks configuration changes and maintains a history of up to 7 years. Config also provides aggregated view of resource configuration and compliance status across accounts and regions. If you are using Amazon Web Services Config rules, Amazon Web Services Config continuously evaluates your Amazon Web Services resource configurations for desired settings. Depending on the rule, Amazon Web Services Config will evaluate your resources either in response to configuration changes or periodically. If a resource violates the conditions of a rule, Amazon Web Services Config flags the resource and the rule as noncompliant. Refer getting started with Amazon Web Services Config for more details. Figure 1 depicts the workflow of Amazon Web Services Config

Figure 1. Workflow of Amazon Web Services Config

Solution overview:

The solution in this blog post includes a serverless architecture for running a query against Amazon Web Services Config aggregator . An Amazon EventBridge Rule triggers an Amazon Web Services Lambda function at the start of every week to find the list of noncompliant resources from noncompliant rules that  are not remediated over a period of 30 days.
The following figure provides the architecture for our solution’s  end-to-end flow:

Figure 2. Solution workflow diagram

1. The Lambda function will run a query against Amazon Web Services Config, thereby getting the list of noncompliant resources over a period of 30 days.
2. The Lambda function will generate a CSV report and send it via email using Amazon Simple Email Service (Amazon SES)

Prerequisites:

Before getting started, make sure that you have a basic understanding of the following:

• Amazon EventBridge rule that runs on a schedule
• Amazon Web Services Config
• Amazon Web Services Config Multi-Account Multi-Region Data Aggregation
• Amazon Web Services Lambda Function
• Python and Boto3 .
• CDK environments .

You will also need to configure the environment

1. Amazon Web Services Config and Amazon Web Services Config recorder

• Enable Amazon Web Services Config in all the accounts in your Amazon Web Services organization. You can use Quick Setup a capability of Amazon Web Services Systems Manager , to help with the setup of the Amazon Web Services Config recorder. Config recorder will detect changes in your resource configurations.

2. To Setup an Aggregator using the Amazon Web Services Console

• Once the Amazon Web Services Config Aggregator is configured, note your Amazon Web Services Config aggregator name.

3. To Setup an Amazon Simple Email Service

• After the setup of Amazon SES, note the Amazon SES arn and email address of sender
• Note: You will need to enable SES production access if you want to send emails to an unverified address in SES.

4. Make sure you have the latest CDK CLI installed
5. Make sure below two libraries exists

python -m pip install aws-cdk-lib
pip install async-timeout

Solution Walkthrough

The Python code in this post was written using the  Amazon Web Services Cloud Development Kit (Amazon Web Services CDK) . To view the code, see the associated  GitHub repository . If you’re unfamiliar with Amazon Web Services CDK, then see  Getting started with Amazon Web Services CDK .

1. You will need to provide following parameter values for the stack:

• • Aggregator – Name of Amazon Web Services Config Aggregator.
  • SESarn – The Amazon SES arn.
  • Recipient – Email recipient that will get the CSV report.
  • Sender – Email sender as configured on Amazon SES.
  • Weekday- Day-of-week 1-7 or SUN-SAT that the Lambda will run.
  • Hour – The hour (UTC) that the Lambda will run.
  • Minute – The minute (UTC) that the Lambda will run.

2. To get the noncompliant rules for aggregator.

• In this blog post,  we first  use the describe_aggregate_compliance_by_config_rules() API to get a  list of all noncompliant rules captured by the aggregator.

3. To get list of noncompliant resources.

• For each noncompliant rule captured, we run a query to get a  list of noncompliant resources and the recorded time for their noncompliant status using get_aggregate_compliance_details_by_config_rule API.

4. From the noncompliant list of resources, we further filter them  using a ‘Recorded Time’ > 30 days parameter
5. We create a CSV file with the list of non-complaint resources over a period of 30 days with details and send an email to recipient with CSV file as an attachment.

Solution Deployment:

1. Make sure you are logged in to the Amazon Web Services management console , and have configured your Amazon Web Services CLI credentials .
2. Clone the solution repository

git clone https://github.com/aws-samples/Config-report

3. Navigate to the cdk directory of the cloned repository

cd Config-Report/config_report

4. Run cdk bootstrap

cdk bootstrap

5. Deploy the Config Report solution

cdk deploy ConfigReport --parameters Aggregator=<name of the aggregator> 
--parameters Recipient=<Recipient email address > --parameters Sender=<Sender
email address > --parameters Weekday=<Day of Week> --parameters Hour=<Hour of
the day> --parameters Minute=<Minute of the day> --parameters SESarn=<SES ARN> 

The “cdk deploy” command adds a new Amazon Web Services CloudFormation template, which creates a Lambda Function with a weekly trigger. At the end of the deployment, the Lambda function will be triggered automatically based on the weekday of your choice. As a result, you should get the CSV report in the recipient mail box.

Cleanup

To avoid recurring charges, and to clean up all the resources after trying the solution outlined in this post, perform the following:

1. Delete the Amazon Web Services Config Aggregator .
2. Delete the Amazon SES .
3. Use cdk cli with the cdk destroy command to delete the CloudFormation Stack.

Are you sure you want to delete: cdk-stack (y/n)? y
cdk-stack: destroying.......
DELETION OF RESOURCES....

 ✅  cdk-stack: destroyed

It will not delete the stack that the bootstrap command created. If you want to delete that as well, you can do it from the Amazon Web Services Console .

Conclusion

In this blog post we demonstrated a sample solution that can help you setup reports for resources that have been noncompliant for more than 30 days. You can also modify the code to run this solution at a different frequency, or to run different queries.

About the author: