Speeding Up IAMTrail: One Boto3 Process Instead of 1,500 CLI Invocations

Sat, 04 Apr 2026 07:37:00 +0200

The 46-Minute Problem

The IAMTrail detection engine fetches ~1,500 AWS managed IAM policies every run. The original approach was pure bash:

aws iam list-policies --output json |
    jq -cr '...' |
    xargs -P 16 -n3 sh -c \
      'aws iam get-policy-version --policy-arn $1 --version-id $2 | jq --indent 4 . > "policies/$3"' sh

Looks fine, right? Except each iteration spawns a full AWS CLI process. That means a fresh Python runtime, boto3 import, credential resolution, one single HTTP call, then exit. Times 1,500.

IAMTrail.com: The AWS Managed Policy Archive (Evolved from MAMIP)

Thu, 23 Oct 2025 13:37:00 +0200

📜 The Origins

Back in 2019, Scott Piper started a GitHub repository to track changes to AWS Managed Policies. It was a simple setup, manually triggered but it worked well and was incredibly useful. Using git diff or the Github.com UI, Scott and the community could easily see how policies evolved over time.

At that time, AWS didn’t publicly share the changes made to managed policies by the various “two-pizza” product teams inside AWS.

Tools on zoph.me

Speeding Up IAMTrail: One Boto3 Process Instead of 1,500 CLI Invocations

The 46-Minute Problem

IAMTrail.com: The AWS Managed Policy Archive (Evolved from MAMIP)

📜 The Origins