Skip to content

eugeneyan/python-collab-template

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.

Tests codecov

python-collab-template

Repository for How to set up a Python Repo for Automation and Collaboration.

Quickstart

# Clone this repo and change directory
git clone git@github.com:eugeneyan/python-collab-template.git
cd python-collab-template

# Built docker image
make setup

# Alternatively, you can create a python virtualenv on your local machine (-B might be needed to execute)
make setup-venv -B

# Run the suite of tests and checks on docker
make run-checks

# Alternatively, run the suite of tests and checks on your local machine
make checks

Make a pull request to this repo to see the checks in action 😎

Here's a sample pull request which initially failed ❌ the checks, and then passed βœ….

Running our checks

In it, we cover the following aspects of setting up a python project, including:

Unit Tests

@pytest.fixture
def lowercased_df():
    string_col = ['futrelle, mme. jacques heath (lily may peel)',
                  'backstrom, major. karl alfred (maria mathilda gustafsson)']
    df_dict = {'string': string_col}
    df = pd.DataFrame(df_dict)
    return df

def test_extract_title(lowercased_df):
    result = extract_title(lowercased_df, col='string')
    assert result['title'].tolist() == ['mme', 'ms', 'mr', 'lady', 'major']


def test_extract_title_with_replacement(lowercased_df):
    title_replacement = {'mme': 'mrs', 'ms': 'miss', 'lady': 'rare', 'major': 'rare'}
    result = extract_title(lowercased_df, col='string', replace_dict=title_replacement)
    assert result['title'].tolist() == ['mrs', 'miss', 'mr', 'rare', 'rare']
$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.8.2, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /Users/eugene/projects/python-collaboration-template/tests/data_prep
collected 2 items

test_categorical.py::test_extract_title PASSED                           [ 50%]
test_categorical.py::test_extract_title_with_replacement PASSED          [100%]

============================== 2 passed in 0.30s ===============================

Code Coverage

$ pytest --cov=src
============================= test session starts ==============================
platform darwin -- Python 3.8.2, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /Users/eugene/projects/python-collaboration-template
plugins: cov-2.10.0
collected 9 items

tests/data_prep/test_categorical.py ....                                 [ 44%]
tests/data_prep/test_continuous.py .....                                 [100%]

---------- coverage: platform darwin, python 3.8.2-final-0 -----------
Name                           Stmts   Miss  Cover
--------------------------------------------------
src/__init__.py                    0      0   100%
src/data_prep/__init__.py          0      0   100%
src/data_prep/categorical.py      12      0   100%
src/data_prep/continuous.py       11      0   100%
--------------------------------------------------
TOTAL                             23      0   100%

============================== 9 passed in 0.49s ===============================

Linting

$ pylint src.data_prep.categorical --reports=y
************* Module src.data_prep.categorical
src/data_prep/categorical.py:20:0: C0330: Wrong continued indentation (add 9 spaces).
                        df[title_col].map(replace_dict),
                        ^        | (bad-continuation)
src/data_prep/categorical.py:21:0: C0330: Wrong continued indentation (add 9 spaces).
                        df[title_col])
                        ^        | (bad-continuation)
src/data_prep/categorical.py:16:12: W1401: Anomalous backslash in string: '\.'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
src/data_prep/categorical.py:1:0: C0114: Missing module docstring (missing-module-docstring)
src/data_prep/categorical.py:5:0: C0116: Missing function or method docstring (missing-function-docstring)
src/data_prep/categorical.py:9:0: C0116: Missing function or method docstring (missing-function-docstring)
src/data_prep/categorical.py:14:0: C0116: Missing function or method docstring (missing-function-docstring)

Report
======
12 statements analysed.

...

Messages
--------
+------------------------------+------------+
|message id                    |occurrences |
+==============================+============+
|missing-function-docstring    |3           |
+------------------------------+------------+
|bad-continuation              |2           |
+------------------------------+------------+
|missing-module-docstring      |1           |
+------------------------------+------------+
|anomalous-backslash-in-string |1           |
+------------------------------+------------+

-----------------------------------
Your code has been rated at 4.17/10

Type Checking

$ mypy src
src/data_prep/continuous.py:23: error: Incompatible types in assignment (expression has type "str", variable has type "float")
Found 1 error in 1 file (checked 4 source files)
$ mypy src
Success: no issues found in 4 source files

Wrapping it in a Makefile

clean-pyc:
	find . -name '*.pyc' -exec rm -f {} +
	find . -name '*.pyo' -exec rm -f {} +
	find . -name '*~' -exec rm -f {} +
	find . -name '__pycache__' -exec rm -fr {} +

clean-test:
	rm -f .coverage
	rm -f .coverage.*

clean: clean-pyc clean-test

test: clean
	. .venv/bin/activate && py.test tests --cov=src --cov-report=term-missing --cov-fail-under 95

GitHub Actions with each git push

# .github/workflows/tests.yml
name: Tests
on: push
jobs:
  tests:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v1
      with:
        python-version: 3.8
        architecture: x64
    - run: make setup
    - run: make check
    - run: bash <(curl -s https://codecov.io/bash)

πŸ‘‰ View the article for the walkthrough.

Todo

  • Update requirements.txt to use poetry

About

πŸ›  Python project template with unit tests, code coverage, linting, type checking, Makefile wrapper, and GitHub Actions.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published