How to deal with custom recorder of AWS Config?

Disclaimer: I’m not a REGEX expert 😄

Lately, I was working for one of my customers on a custom configuration of AWS Config recorder.

My customer wanted to record using AWS Config All resources except a few of them:

  • 'AWS::EC2::Subnet'
  • 'AWS::EC2::VPC'
  • 'AWS::EC2::SecurityGroup'

Unfortunately, the AWS API and Console do not allow you to do this, you should cherry-pick manually which resource you want to record.

The trade-off of this method is that if a new AWS Config resource type came out, it won’t be recorded until you manually select it in your AWS Config recorder setting.

To deal with this, I’ve created a very simple python script that is using AWS documentation web-scraping to extract supported resource types, and then apply all resources except those which are blacklisted by my customer.

The idea is to schedule this script once a week to be up-to-date.

import boto3
import re
from urllib.request import urlopen
import logging


# Purpose:
# Activate Custom AWS Record for AWS Config
# Supported resource type:
# Scraping AWS Docs using:

# Get information about the current regional config recorder: aws configservice describe-configuration-recorders --region eu-west-1

# Logging
root = logging.getLogger()
if root.handlers:
    for handler in root.handlers:
logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s',level=logging.INFO)

recorder_name = "<AWS Config recorder name>"
role_arn = "<role arn used for AWS Config>"

# Put here the AWS Config resources type to exclude
exclusion_list = [

def get_config_resources():
    url = ""
    page = urlopen(url)
    html ="utf-8")

    # Target format: AWS::ApiGateway::Stage
    pattern = "AWS::.*"
    match_results = re.findall(pattern, html)
    cleaned_list = []
    count = 0

    for result in match_results:
        # remove HTML tags
        results = re.sub("<.*?>", "", result) 
        # remove ending *
        results = results.replace("*", "")
        # remove space
        results = results.replace(" ", "")
        # remove long items (sentences)
        if len(results) >= 60:
        # distinct list while preserving order
        # Count items
        count += 1
        # Create the target cleaned list
        cleaned_list.append(results)"Scraped Config supported resources: %s", count)

    return cleaned_list

def apply_custom_recorder(config_resources):
    # Remove excluded resources from the globql list
    result_list = list(set(config_resources) - set(exclusion_list))

    # counter
    count_result = 0

    # Count resulted number of resource types (minus excluded types)
    for type in result_list:
        count_result += 1"result_types: %s", count_result)

    client = boto3.client('config')

        r = client.put_configuration_recorder(
                    'name': recorder_name,
                    'roleARN': role_arn,
                    'recordingGroup': {
                        'allSupported': False,
                        'includeGlobalResourceTypes': False,
                        'resourceTypes': result_list
    except Exception as e:
        logging.error(e)"Response: %s", r)

if __name__ == "__main__":
    config_resources = get_config_resources()

Gist version if you want to contribute.

I think this approach could help someone else, so I’m sharing it with you. Don’t hesitate to comment, and enhance it as you like.

That’s all folks!


comments powered by Disqus