Skip to content
This repository was archived by the owner on Aug 24, 2023. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
5d92f68
Add support to run this package as a module
mcdonnnj Sep 28, 2020
b072dc4
Change execution status of example.py
mcdonnnj Oct 6, 2020
5c83a56
Add support for Python 3.9
mcdonnnj Oct 6, 2020
6a4164d
Bump build workflow python-version to 3.9
felddy Oct 7, 2020
c9c00f1
Replace deprecated set-env workflow syntax
felddy Oct 7, 2020
7389a1b
Enable weekly dependabot scans of github-actions and pip
felddy Oct 7, 2020
0923631
Merge pull request #52 from cisagov/improvement/workflows
felddy Oct 7, 2020
8fdada3
Merge https://github.com/cisagov/skeleton-generic into lineage/skeleton
Oct 7, 2020
d79989a
Add testing for __main__.py in pursuit of glorious 100% coverage
mcdonnnj Oct 7, 2020
1f98e8e
Merge pull request #53 from cisagov/lineage/skeleton
mcdonnnj Oct 7, 2020
9d7bd0d
Merge branch 'develop' into improvements/add_py39_support
mcdonnnj Oct 7, 2020
b4ed5ce
Merge pull request #52 from cisagov/improvements/add_py39_support
mcdonnnj Oct 7, 2020
cf2abe4
Merge branch 'develop' into improvement/add_run_as_module_support
mcdonnnj Oct 7, 2020
0db80d1
Merge pull request #51 from cisagov/improvement/add_run_as_module_sup…
mcdonnnj Oct 7, 2020
db59fc1
Enable CodeQL scanning for Python.
felddy Oct 8, 2020
e14ecde
Add link to how to set up a new repo with skeletons
hillaryj Oct 9, 2020
57d927d
Add line breaks for linting
hillaryj Oct 9, 2020
62b3de6
Merge pull request #55 from cisagov/improvement/codeql
felddy Oct 9, 2020
b5ef97d
Remove seed-isort-config pre-commit hook
mcdonnnj Oct 19, 2020
de409db
Use local filename in README link
mcdonnnj Oct 19, 2020
59eef4c
Run pre-commit autoupdate
mcdonnnj Oct 19, 2020
a9ef644
Merge pull request #53 from cisagov/add-how-to-skeleton
mcdonnnj Oct 22, 2020
1928db6
Merge branch 'develop' into pre-commit_updates
mcdonnnj Oct 22, 2020
4d88e06
Merge pull request #54 from cisagov/pre-commit_updates
mcdonnnj Oct 22, 2020
d5c31a9
Resolve Lineage conflicts
mcdonnnj Oct 22, 2020
a1c3228
Resolve Lineage conflicts
mcdonnnj Oct 22, 2020
73a7dcd
Add explanation for the noqa used in one of the tests
mcdonnnj Oct 22, 2020
72715e3
Change a test name to be more descriptive
mcdonnnj Oct 22, 2020
919460c
Change repo for the prettier pre-commit hook
mcdonnnj Oct 22, 2020
e44762c
Merge pull request #55 from cisagov/migrate_prettier_hook
mcdonnnj Oct 22, 2020
8ccc56c
Merge pull request #17 from cisagov/lineage/skeleton
mcdonnnj Oct 22, 2020
e08b5bc
Merge https://github.com/cisagov/skeleton-generic into lineage/skeleton
Oct 22, 2020
0f6efe0
Merge pull request #56 from cisagov/lineage/skeleton
mcdonnnj Oct 22, 2020
7083e84
Resolve Lineage conflicts
mcdonnnj Oct 22, 2020
e783b26
Merge pull request #18 from cisagov/lineage/skeleton
mcdonnnj Oct 23, 2020
468863e
Merge github.com:cisagov/skeleton-aws-lambda into lineage/skeleton
mcdonnnj Oct 23, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---

version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"

- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
11 changes: 6 additions & 5 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.9
- name: Store installed Python version
run: |
echo "::set-env name=PY_VERSION::"\
"$(python -c "import platform;print(platform.python_version())")"
echo "PY_VERSION="\
"$(python -c "import platform;print(platform.python_version())")" \
>> $GITHUB_ENV
- name: Cache linting environments
uses: actions/cache@v2
with:
Expand All @@ -48,7 +49,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
python-version: [3.6, 3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
Expand Down Expand Up @@ -82,7 +83,7 @@ jobs:
needs: [lint, test]
strategy:
matrix:
python-version: [3.6, 3.7, 3.8]
python-version: [3.6, 3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
Expand Down
76 changes: 76 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
name: "CodeQL"

on:
push:
# Push on all branches
# branches: [develop]
pull_request:
# The branches below must be a subset of the branches above
branches: [develop]
schedule:
- cron: '0 14 * * 6'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
# Override automatic language detection by changing the below list
# Supported options are ['csharp', 'cpp', 'go', 'java', 'javascript',
# 'python']
language: ['python']
# Learn more...
# https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection

steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2

# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a
# config file. By default, queries listed here will override any
# specified in a config file. Prefix the list here with "+" to use
# these queries and those in the config file. queries:
# ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or
# Java). If this step fails, then you should remove it and run the build
# manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following
# three lines and modify them (or add more) to build your code if your
# project uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
5 changes: 0 additions & 5 deletions .isort.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,5 @@ import_heading_stdlib=Standard Python Libraries
import_heading_thirdparty=Third-Party Libraries
import_heading_firstparty=cisagov Libraries

# Should be auto-populated by seed-isort-config hook
known_third_party=boto3,botocore,docopt,pymongo,pytest,setuptools
# These must be manually set to correctly separate them from third party libraries
known_first_party=

# Run isort under the black profile to align with our other Python linting
profile=black
22 changes: 9 additions & 13 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,21 +27,21 @@ repos:
- id: requirements-txt-fixer
- id: trailing-whitespace
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.23.2
rev: v0.24.0
hooks:
- id: markdownlint
args:
- --config=.mdl_config.json
- repo: https://github.com/adrienverge/yamllint
rev: v1.24.2
rev: v1.25.0
hooks:
- id: yamllint
- repo: https://github.com/detailyang/pre-commit-shell
rev: 1.0.5
hooks:
- id: shell-lint
- repo: https://gitlab.com/pycqa/flake8
rev: 3.8.3
rev: 3.8.4
hooks:
- id: flake8
additional_dependencies:
Expand Down Expand Up @@ -70,21 +70,17 @@ repos:
rev: 20.8b1
hooks:
- id: black
- repo: https://github.com/asottile/seed-isort-config
rev: v2.2.0
hooks:
- id: seed-isort-config
- repo: https://github.com/timothycrosley/isort
rev: 5.5.0
rev: 5.6.4
hooks:
- id: isort
- repo: https://github.com/ansible/ansible-lint.git
rev: v4.3.4
rev: v4.3.5
hooks:
- id: ansible-lint
# files: molecule/default/playbook.yml
- repo: https://github.com/antonbabenko/pre-commit-terraform.git
rev: v1.37.0
rev: v1.43.0
hooks:
- id: terraform_fmt
# There are ongoing issues with how this command works. This issue
Expand All @@ -107,11 +103,11 @@ repos:
rev: v2.0.0
hooks:
- id: docker-compose-check
- repo: https://github.com/prettier/prettier
rev: 2.1.1
- repo: https://github.com/prettier/pre-commit
rev: v2.1.2
hooks:
- id: prettier
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.782
rev: v0.790
hooks:
- id: mypy
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ using this tool.

## Contributing ##

We welcome contributions! Please see [here](CONTRIBUTING.md) for
We welcome contributions! Please see [`CONTRIBUTING.md`](CONTRIBUTING.md) for
details.

## License ##
Expand Down
5 changes: 5 additions & 0 deletions fdi/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""Code to run if this package is used as a Python module."""

from .findings_data_import import main

main()
6 changes: 0 additions & 6 deletions fdi/findings_data_import.py
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
#!/usr/bin/env python3

"""findings_data_import: A tool for extracting, transforming and loading JSON data to Mongo.

The source data is a JSON file stored in an AWS S3 bucket.
Expand Down Expand Up @@ -325,7 +323,3 @@ def main():
logging.shutdown()

return 0 if result else -1


if __name__ == "__main__":
main()
2 changes: 1 addition & 1 deletion requirements-test.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
--editable .[test]
--requirement requirements.txt
-e .[test]
3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ def package_vars(version_file):
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
],
python_requires=">=3.6",
# What does your project relate to?
Expand All @@ -84,6 +85,6 @@ def package_vars(version_file):
"pytest",
]
},
# Conveniently allows one to run the CLI tool as `example`
# Conveniently allows one to run the CLI tool as `fdi`
entry_points={"console_scripts": ["fdi = fdi.findings_data_import:main"]},
)
17 changes: 17 additions & 0 deletions tests/test_findings_data_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,23 @@ def test_stdout_version(capsys):
), "standard output by '--version' should agree with module.__version__"


def test_running_as_module(capsys):
"""Verify that the __main__.py file loads correctly."""
with pytest.raises(SystemExit):
with patch.object(sys, "argv", ["bogus", "--version"]):
# F401 is a "Module imported but unused" warning. This import
# emulates how this project would be run as a module. The only thing
# being done by __main__ is importing the main entrypoint of the
# package and running it, so there is nothing to use from this
# import. As a result, we can safely ignore this warning.
# cisagov Libraries
import fdi.__main__ # noqa: F401
captured = capsys.readouterr()
assert (
captured.out == f"{PROJECT_VERSION}\n"
), "standard output by '--version' should agree with module.__version__"


@pytest.mark.skipif(
RELEASE_TAG in [None, ""], reason="this is not a release (RELEASE_TAG not set)"
)
Expand Down