Speeding Up IAMTrail: One Boto3 Process Instead of 1,500 CLI Invocations
The 46-Minute Problem The IAMTrail detection engine fetches ~1,500 AWS managed IAM policies every run. The original approach was pure bash: 1 2 3 4 aws iam list-policies --output json | jq -cr '...' | xargs -P 16 -n3 sh -c \ 'aws iam get-policy-version --policy-arn $1 --version-id $2 | jq --indent 4 . > "policies/$3"' sh Looks fine, right? Except each iteration spawns a full AWS CLI process. That means a fresh Python runtime, boto3 import, credential resolution, one single HTTP call, then exit. Times 1,500. ...