Automate your Steampipe AWS configuration with AWS Organizations

This time we bring you our first public tool!

If you use Steampipe with your AWS organization, this will interest you. We have built this tool to automate the creation and maintenance of the configuration files necessary to work with all the accounts in an AWS organization.

Steampipe is a tool that allows you to perform queries on APIs, among other things, using SQL. One of its great uses is with AWS, allowing you to obtain information about your resources using SQL.

If you want to use it with multiple AWS accounts, you will need to modify the file ~/.steampipe/config/aws.spc, adding each account, and the file ~/.aws/credentials, adding the necessary profiles for each account. You can find all the details in their documentation.

If your AWS organization changes frequently, keeping these files updated can be tedious. Steampipe has scripts to automate this in the steampipe-samples repository, but we have decided to take this automation to the next level and add some additional features.

Requirements

To use the tool, you will need:

AWS credentials with the following IAM permissions:

"organizations:ListAccounts",
"organizations:ListParents",
"organizations:ListTagsForResource"

An IAM Role created in all accounts of your organization with the required permissions for Steampipe queries, for example, the managed policy arn:aws:iam::aws:policy/SecurityAudit. For more details on how to do this, you can check out our post Deploy IAM Roles across an AWS Organization as code.

Features

Automates the generation of the files ~/.steampipe/config/aws.spc and ~/.aws/credentials.
Retrieves all accounts from your AWS Organization.
Ability to ignore Organizational Units (OUs).
Supports IMDSv2 on EC2 and ECS, as well as local environments.
Template for adding custom aggregators.
Create aggregators based on account tags.
Enable/Disable Steampipe import schema.

How does it work?

Retrieves a complete list of all names and IDs of the active accounts in the organization.
If any OU to be avoided has been specified, it checks the accounts within it to avoid generating their connections.
Obtains the tags associated with each account.
Normalizes account names to lowercase, replacing spaces and dashes with _.
Generates the configuration files based on a template.

How to use it?

./steampipe_config_generator -role my-org-role-name

If you need to run it on EC2, you must use the flag -credential Ec2InstanceMetadata. If running it on ECS, you should use -credential EcsContainer.

Important

Check the repository steampipe-config-generator for all available options. There, you will always find the updated information.

Tip

You can run the script before starting Steampipe to always keep your connections updated.

docker-entrypoint.sh

#!/usr/bin/env bash

./config_generator -role my-org-role-name -credential EcsContainer

exec "$@"

Dockerfile

...
ENTRYPOINT [ "/bin/bash", "docker-entrypoint.sh" ]
CMD [ "steampipe", "service", "start", "--foreground"]

Custom Aggregators

The file aws_connections.tmpl is used as a template to generate the connections. You can edit it to add the desired aggregators, such as:

Aggregator based on your AWS account names. This example will create an aggregator for all your AWS accounts whose names start with Sandbox:
```
connection "aws_sandbox" {
  plugin      = "aws"
  type        = "aggregator"
  connections = ["aws_sandbox_*"]
}
```
Remember that all account names should be in lowercase, with spaces and dashes replaced by _.

Aggregator based on your Account tags. This example will create an aggregator for all your AWS accounts that have the tag team:engineering:

{{ $teamEng := index .Tags "team,engineering" }}
connection "aws_engineering_team" {
  plugin      = "aws"
  type        = "aggregator"
  connections = [{{- range $index, $name := $teamEng -}}{{ if $index }}, {{ end }}"aws_{{ $name }}"{{- end }}]
}

And that's all folks! If you have any questions or comments, feel free to reach out to us.

Saludos, and may the force be with you.