Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file removed .editorconfig
Empty file.
4 changes: 2 additions & 2 deletions .github/CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,11 +117,11 @@ This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].

Community Impact Guidelines were inspired by
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].

For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
at [https://www.contributor-covenant.org/translations][translations].

[homepage]: https://www.contributor-covenant.org
Expand Down
38 changes: 38 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,41 @@ jobs:

- name: Run tests
run: make test

analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 1

- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.9

- name: Install Poetry
uses: snok/install-poetry@v1.1.6
with:
virtualenvs-create: false
virtualenvs-in-project: true

- name: Restore cache venv
uses: syphar/restore-virtualenv@v1
id: cache-venv

- name: Initialize CodeQL
uses: github/codeql-action/init@v1.0.3
with:
languages: python
setup_python_dependencies: false

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1.0.3
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@ dist
.pnp.*

# -------------------------------------------------------------------------------------
# Python
# Python
# -------------------------------------------------------------------------------------

# Byte-compiled / optimized / DLL files
Expand Down Expand Up @@ -1101,3 +1101,7 @@ $RECYCLE.BIN/

# Windows shortcuts
*.lnk

# Misc

*.bak
20 changes: 10 additions & 10 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
# pre-commit run --all-files
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.4.0
rev: v4.0.1
hooks:
- id: check-ast
- id: check-json
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: debug-statements
- id: detect-aws-credentials
args: ["--allow-missing-credentials"]
- id: detect-private-key
- id: end-of-file-fixer
- id: mixed-line-ending
- id: no-commit-to-branch
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 21.6b0
rev: 21.11b1
hooks:
- id: black
- repo: https://github.com/asottile/blacken-docs
rev: v1.10.0
rev: v1.12.0
hooks:
- id: blacken-docs
additional_dependencies: [black==21.6b0]
- repo: https://github.com/flakehell/flakehell
rev: v.0.8.0
hooks:
- name: Run flakehell static analysis tool
id: flakehell
# Disable flakehell for now due to import bug with newer flake8
# https://github.com/flakehell/flakehell/issues/22
# - repo: https://github.com/flakehell/flakehell
# rev: v.0.9.0
# hooks:
# - name: Run flakehell static analysis tool
# id: flakehell
11 changes: 10 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,23 @@ the unreleased section to the section for the new release.

No unreleased changes.

## [0.1.1] - 2021-11-23

### Added

- Adds support for generating a Graphviz diagram of an Organization with the new
`OrganizationDataBuilder.to_dot()` function
- Adds `DOT` as a supported output format for the `organization dump-all` command
- Adds models for AWS Config notifications
- Adds models for SQS and SNS messages
- Adds methods to ModelBase to allow (de)serializing JSON or YAML strings
- Adds ModelBase.from_dict() to initialize a model from a dict using dacite
- Adds CodeQL analysis workflow for GitHub Actions

### Changed

- breaking: Renames `organization dump-json` CLI command to `organization dump-all`
- Moves buiders into the models namespace

## [0.1.0-beta2] - 2021-06-16

Expand Down Expand Up @@ -78,7 +86,8 @@ Initial alpha release
These Markdown anchors provide a link to the diff for each release. They should be
updated any time a new release is cut.
-->
[Unreleased]: https://github.com/timoguin/aws-org-tools-py/compare/v0.1.0-beta-2...HEAD
[Unreleased]: https://github.com/timoguin/aws-org-tools-py/compare/v0.1.1...HEAD
[0.1.1]: https://github.com/timoguin/aws-org-tools-py/compare/v0.1.0-beta2...v0.1.1
[0.1.0-beta2]: https://github.com/timoguin/aws-org-tools-py/compare/v0.1.0-beta1...v0.1.0-beta2
[0.1.0-beta1]: https://github.com/timoguin/aws-org-tools-py/compare/v0.1.0-alpha4...v0.1.0-beta1
[0.1.0-alpha4]: https://github.com/timoguin/aws-org-tools-py/releases/tag/v0.1.0-alpha4
7 changes: 4 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@ shellcmd: ${VENV_DIR}
lint: ${VENV_DIR}
@echo "Running the black code formatter"
@poetry run black ${ARGS} .
@echo "Running flakehell plugins"
@poetry run flakehell lint aws_data_tools
# flakehell currently disabled, see .pre-commit-config.yaml notesk
# @echo "Running flakehell plugins"
# @poetry run flakehell lint aws_data_tools

.PHONY: lint-docs
lint-docs: ${VENV_DIR}
Expand Down Expand Up @@ -72,7 +73,7 @@ build: ${VENV_DIR}
.PHONY: test ## Run the test suite
test: ${VENV_DIR}
@echo "Running test suite"
@poetry run pytest --suppress-no-test-exit-code
@poetry run pytest --cov ${OPTS} ${ARGS}

# Ensures the Python venv exists and has dependencies installed
${VENV_DIR}:
Expand Down
27 changes: 15 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# AWS Data Tools

<!-- Badges -->
[![Actions Status][gh-actions-badge]][gh-actions-link]
[![Actions CI Status][gh-actions-ci-badge]][gh-actions-ci-link]
[![Actions CodeQL Status][gh-actions-codeql-badge]][gh-actions-codeql-link]
[![PyPI][pypi-badge]][pypi-link]
[![License][license-badge]][license-link]

Expand Down Expand Up @@ -32,7 +33,7 @@ To dump a data representation of an AWS Organization, you can do the following u
the builder:

```python
from aws_data_tools.builders.organizations import OrganizationDataBuilder
from aws_data_tools.models.organizations import OrganizationDataBuilder

odb = OrganizationDataBuilder(init_all=True)
organization = odb.as_json()
Expand All @@ -41,7 +42,7 @@ organization = odb.as_json()
Here is how to do the same thing with the CLI:

```
$ awsdata organization dump-json
$ awsdata organization dump-all
```

## Usage
Expand All @@ -54,9 +55,9 @@ abstract some of these operations.
### Builders

While it is possible to directly utilize and interact with the data models, probably
the largest benefit is the [builders](aws_data_tools/builders) package. It abstracts
any API operations and data transformations required to build data models. The models
can then be serialized to dicts, as well as JSON or YAML strings.
the largest benefit are the builders. They abstract any API operations and data
transformations required to build data models. The models can then be serialized to
dicts, as well as DOT, JSON, or YAML strings.

A full model of an AWS Organization can be constructed using the
`OrganizationDataBuilder` class. It handles recursing the organizational tree and
Expand All @@ -67,9 +68,9 @@ The simplest example pulls all supported organizational data and creates the rel
data models:

```python
from aws_data_tools.builders.organizations import OrganizationDataBuilder as odb
from aws_data_tools.models.organizations import OrganizationDataBuilder

org = odb(init_all=True)
odb = OrganizationDataBuilder(init_all=True)
```

Note that this makes many API calls to get this data. For example, every OU, policy,
Expand All @@ -84,9 +85,9 @@ requires 316 API calls! That's why this library was created.
For more control over the process, you can init each set of components as desired:

```python
from aws_data_tools.builders.organizations import OrganizationDataBuilder as odb
from aws_data_tools.models.organizations import OrganizationDataBuilder

org = odb()
org = OrganizationDataBuilder()
org.init_connection()
org.init_organization()
org.init_root()
Expand Down Expand Up @@ -274,8 +275,10 @@ View the [Contributing Guide](.github/CONTRIBUTING.md) to learn about giving bac


<!-- Markown anchors -->
[gh-actions-badge]: https://github.com/timoguin/aws-data-tools-py/actions/workflows/ci.yml/badge.svg
[gh-actions-link]: https://github.com/timoguin/aws-data-tools-py/actions
[gh-actions-ci-badge]: https://github.com/timoguin/aws-data-tools-py/actions/workflows/ci.yml/badge.svg
[gh-actions-ci-link]: https://github.com/timoguin/aws-data-tools-py/actions/workflows/ci.yml
[gh-actions-codeql-badge]: https://github.com/timoguin/aws-data-tools-py/actions/workflows/codeql-analysis.yml/badge.svg
[gh-actions-codeql-link]: https://github.com/timoguin/aws-data-tools-py/actions/workflows/codeql-analysis.yml
[license-badge]: https://img.shields.io/github/license/timoguin/aws-data-tools-py.svg
[license-link]: https://github.com/timoguin/aws-data-tools-py/blob/main/LICENSE
[pypi-badge]: https://badge.fury.io/py/aws-data-tools.svg
Expand Down
3 changes: 3 additions & 0 deletions aws_data_tools/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
"""
A library for working with data from AWS APIs
"""
# flake8: noqa: F401

from . import client, models, utils

__VERSION__ = "0.1.0-beta2"

Expand Down
Empty file.
Loading