Drop python 3.7 + Setup Poetry + CodeCov + Pre-Commit (#300)

* Setup Poetry + Code Coverage Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Update pre-commit config Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Apply pre-commit hooks to almost all files Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Add a linting step with pre-commit in the CI Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Reduce number of CI runners used for PRs Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Don't test against py3.7 in CI Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Slightly adjust regression files (test moved a bit) Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Add codecov.yml file Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Don't exclude examples and tests for ruff hook Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Run tests with and without extra dependencies Signed-off-by: Fabrice Normandin <normandf@mila.quebec> * Add poetry check hook in pre-commit config Signed-off-by: Fabrice Normandin <normandf@mila.quebec> --------- Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
lebrice · Jan 31, 2024 · 7e1edcd · 7e1edcd
1 parent 8bec502
commit 7e1edcd
Show file tree

Hide file tree

Showing 121 changed files with 1,282 additions and 3,529 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -1,26 +1,67 @@
 name: Python package
 
-on: [push, pull_request]
+on:
+  push:
+    branches: [master]
+  pull_request:
 
 jobs:
-  build:
+  linting:
+    name: Run linting/pre-commit checks
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+          cache: 'pip'
+      - run: pip install pre-commit
+      - run: pre-commit --version
+      - run: pre-commit install
+      - run: pre-commit run --all-files
 
+  build:
+    needs: [linting]
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
+      - name: Install poetry
+        run: |
+          python -m pip install --upgrade pip
+          pip install poetry
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v3
+        uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
-          cache: "pip"
-      - name: Install dependencies
+          cache: poetry
+
+      - name: Install base dependencies
+        run: poetry install
+
+      - name: Unit tests with Pytest (no extras)
+        timeout-minutes: 3
         run: |
-          python -m pip install --upgrade pip
-          pip install -e .[all]
-      - name: Unit tests with Pytest
+          poetry run pytest --benchmark-disable --cov=simple_parsing --cov-report=xml --cov-append
+
+
+      - name: Install extra dependencies
+        run: poetry install --all-extras
+
+      - name: Unit tests with Pytest (with extra dependencies)
+        timeout-minutes: 3
         run: |
-          pytest --benchmark-disable
+          poetry run pytest --benchmark-disable --cov=simple_parsing --cov-report=xml --cov-append
+
+
+      - name: Upload coverage reports to Codecov
+        uses: codecov/codecov-action@v3
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+        with:
+          env_vars: PLATFORM,PYTHON
+          name: codecov-umbrella
+          fail_ci_if_error: false
diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml
@@ -5,28 +5,36 @@ name: Upload Python Package
 
 on:
   release:
-    types: [created]
+    types: [published]
   workflow_dispatch: {}
 
 jobs:
-  deploy:
+  publish:
+    strategy:
+      matrix:
+        python-version: [3.9]
+        os: [ubuntu-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install poetry
+        run: |
+          python -m pip install --upgrade pip
+          pip install poetry
 
-    runs-on: ubuntu-latest
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: poetry
 
-    steps:
-    - uses: actions/checkout@v2
-    - name: Set up Python
-      uses: actions/setup-python@v2
-      with:
-        python-version: '3.x'
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install setuptools wheel twine
-    - name: Build and publish
-      env:
-        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
-        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
-      run: |
-        python setup.py sdist bdist_wheel
-        twine upload dist/*
+      - name: Install dependencies
+        run: |
+          poetry install
+          poetry self add "poetry-dynamic-versioning[plugin]"
+          poetry dynamic-versioning enable
+
+      - name: Publish package
+        env:
+          POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_TOKEN }}
+        run: poetry publish --build
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -28,25 +28,23 @@ repos:
 
   - repo: https://github.com/charliermarsh/ruff-pre-commit
     # Ruff version.
-    rev: 'v0.0.261'
+    rev: 'v0.1.14'
     hooks:
+      # Run the linter.
       - id: ruff
-        args: ['--line-length', '99', '--fix']
+        args: ['--line-length', '99', "--select", "I,UP", '--fix']
         require_serial: true
-
-  # python code formatting
-  - repo: https://github.com/psf/black
-    rev: 22.12.0
-    hooks:
-      - id: black
-        args: [--line-length, "99"]
+      # Run the formatter.
+      - id: ruff-format
+        args: ['--line-length', '99']
         require_serial: true
 
   # python docstring formatting
   - repo: https://github.com/myint/docformatter
     rev: v1.5.1
     hooks:
       - id: docformatter
+        exclude: ^test/test_docstrings.py
         args: [--in-place, --wrap-summaries=99, --wrap-descriptions=99]
         require_serial: true
 
@@ -72,6 +70,7 @@ repos:
     hooks:
       - id: mdformat
         args: ["--number"]
+        exclude: ^.github/ISSUE_TEMPLATE/.*\.md$
         additional_dependencies:
           - mdformat-gfm
           - mdformat-tables
@@ -80,6 +79,11 @@ repos:
           # - mdformat-black
         require_serial: true
 
+  - repo: https://github.com/python-poetry/poetry
+    rev: 1.7.0
+    hooks:
+      - id: poetry-check
+        require_serial: true
 
   # word spelling linter
   - repo: https://github.com/codespell-project/codespell

diff --git a/README.md b/README.md
@@ -1,6 +1,5 @@
 ![Build Status](https://github.com/lebrice/SimpleParsing/actions/workflows/build.yml/badge.svg) [![PyPI version](https://badge.fury.io/py/simple-parsing.svg)](https://badge.fury.io/py/simple-parsing)
 
-
 # Simple, Elegant, Typed Argument Parsing <!-- omit in toc -->
 
 `simple-parsing` allows you to transform your ugly `argparse` scripts into beautifully structured, strongly typed little works of art. This isn't a fancy, complicated new command-line tool either, ***this simply adds new features to plain-old argparse!***
@@ -28,11 +27,13 @@ args = parser.parse_args()
 print("foo:", args.foo)
 print("options:", args.options)
 ```
+
 ```console
 $ python examples/demo.py --log_dir logs --foo 123
 foo: 123
 options: Options(log_dir='logs', learning_rate=0.0001)
 ```
+
 ```console
 $ python examples/demo.py --help
 usage: demo.py [-h] [--foo int] --log_dir str [--learning_rate float]
@@ -60,7 +61,6 @@ options: Options = simple_parsing.parse(Options)
 options, leftover_args = simple_parsing.parse_known_args(Options)
 ```
 
-
 ## installation
 
 `pip install simple-parsing`
@@ -70,66 +70,69 @@ options, leftover_args = simple_parsing.parse_known_args(Options)
 ## [API Documentation](https://github.com/lebrice/SimpleParsing/tree/master/docs/README.md) (Under construction)
 
 ## Features
+
 - ### [Automatic "--help" strings](https://github.com/lebrice/SimpleParsing/tree/master/examples/docstrings/README.md)
 
-    As developers, we want to make it easy for people coming into our projects to understand how to run them. However, a user-friendly `--help` message is often hard to write and to maintain, especially as the number of arguments increases.
+  As developers, we want to make it easy for people coming into our projects to understand how to run them. However, a user-friendly `--help` message is often hard to write and to maintain, especially as the number of arguments increases.
 
-    With `simple-parsing`, your arguments and their descriptions are defined in the same place, making your code easier to read, write, and maintain.
+  With `simple-parsing`, your arguments and their descriptions are defined in the same place, making your code easier to read, write, and maintain.
 
 - ### Modular, Reusable, Cleanly Grouped Arguments
 
-    *(no more copy-pasting)*
-
-    When you need to add a new group of command-line arguments similar to an existing one, instead of copy-pasting a block of `argparse` code and renaming variables, you can reuse your argument class, and let the `ArgumentParser` take care of adding relevant prefixes to the arguments for you:
-
-    ```python
-    parser.add_arguments(Options, dest="train")
-    parser.add_arguments(Options, dest="valid")
-    args = parser.parse_args()
-    train_options: Options = args.train
-    valid_options: Options = args.valid
-    print(train_options)
-    print(valid_options)
-    ```
-    ```console
-    $ python examples/demo.py \
-        --train.log_dir "training" \
-        --valid.log_dir "validation"
-    Options(log_dir='training', learning_rate=0.0001)
-    Options(log_dir='validation', learning_rate=0.0001)
-    ```
-
-    These prefixes can also be set explicitly, or not be used at all. For more info, take a look at the [Prefixing Guide](https://github.com/lebrice/SimpleParsing/tree/master/examples/prefixing/README.md)
+  *(no more copy-pasting)*
+
+  When you need to add a new group of command-line arguments similar to an existing one, instead of copy-pasting a block of `argparse` code and renaming variables, you can reuse your argument class, and let the `ArgumentParser` take care of adding relevant prefixes to the arguments for you:
+
+  ```python
+  parser.add_arguments(Options, dest="train")
+  parser.add_arguments(Options, dest="valid")
+  args = parser.parse_args()
+  train_options: Options = args.train
+  valid_options: Options = args.valid
+  print(train_options)
+  print(valid_options)
+  ```
+
+  ```console
+  $ python examples/demo.py \
+      --train.log_dir "training" \
+      --valid.log_dir "validation"
+  Options(log_dir='training', learning_rate=0.0001)
+  Options(log_dir='validation', learning_rate=0.0001)
+  ```
+
+  These prefixes can also be set explicitly, or not be used at all. For more info, take a look at the [Prefixing Guide](https://github.com/lebrice/SimpleParsing/tree/master/examples/prefixing/README.md)
 
 - ### [Argument subgroups](https://github.com/lebrice/SimpleParsing/tree/master/examples/subgroups/README.md)
 
-    It's easy to choose between different argument groups of arguments, with the `subgroups`
-    function!
+  It's easy to choose between different argument groups of arguments, with the `subgroups`
+  function!
 
 - ### [Setting defaults from Configuration files](https://github.com/lebrice/SimpleParsing/tree/master/examples/config_files/README.md)
 
-    Default values for command-line arguments can easily be read from many different formats, including json/yaml!
+  Default values for command-line arguments can easily be read from many different formats, including json/yaml!
 
 - ### [**Easy serialization**](https://github.com/lebrice/SimpleParsing/tree/master/examples/serialization/README.md):
 
-    Easily save/load configs to `json` or `yaml`!.
+  Easily save/load configs to `json` or `yaml`!.
 
 - ### [**Inheritance**!](https://github.com/lebrice/SimpleParsing/tree/master/examples/inheritance/README.md)
 
-    You can easily customize an existing argument class by extending it and adding your own attributes, which helps promote code reuse across projects. For more info, take a look at the [inheritance example](https://github.com/lebrice/SimpleParsing/tree/master/examples/inheritance/inheritance_example.py)
+  You can easily customize an existing argument class by extending it and adding your own attributes, which helps promote code reuse across projects. For more info, take a look at the [inheritance example](https://github.com/lebrice/SimpleParsing/tree/master/examples/inheritance/inheritance_example.py)
 
 - ### [**Nesting**!](https://github.com/lebrice/SimpleParsing/tree/master/examples/nesting/README.md):
 
-    Dataclasses can be nested within dataclasses, as deep as you need!
+  Dataclasses can be nested within dataclasses, as deep as you need!
 
 - ### [Easier parsing of lists and tuples](https://github.com/lebrice/SimpleParsing/tree/master/examples/container_types/README.md) :
-    This is sometimes tricky to do with regular `argparse`, but `simple-parsing` makes it a lot easier by using the python's builtin type annotations to automatically convert the values to the right type for you.
-    As an added feature, by using these type annotations, `simple-parsing` allows you to parse nested lists or tuples, as can be seen in [this example](https://github.com/lebrice/SimpleParsing/tree/master/examples/merging/README.md)
+
+  This is sometimes tricky to do with regular `argparse`, but `simple-parsing` makes it a lot easier by using the python's builtin type annotations to automatically convert the values to the right type for you.
+  As an added feature, by using these type annotations, `simple-parsing` allows you to parse nested lists or tuples, as can be seen in [this example](https://github.com/lebrice/SimpleParsing/tree/master/examples/merging/README.md)
 
 - ### [Enums support](https://github.com/lebrice/SimpleParsing/tree/master/examples/enums/README.md)
 
 - (More to come!)
 
-
 ## Examples:
+
 Additional examples for all the features mentioned above can be found in the [examples folder](https://github.com/lebrice/SimpleParsing/tree/master/examples/README.md)
diff --git a/codecov.yml b/codecov.yml
@@ -0,0 +1,14 @@
+coverage:
+  status:
+    project:
+      default:
+        target: auto  # auto compares coverage to the previous base commit
+        informational: true
+    patch:
+      default:
+        target: 100%
+        informational: true
+
+
+# When modifying this file, please validate using
+# curl -X POST --data-binary @codecov.yml https://codecov.io/validate
diff --git a/docs/Roadmap.md b/docs/Roadmap.md
@@ -1,11 +1,13 @@
 ## Currently supported features:
-* Parsing of attributes of built-in types:
-    * `int`, `float`, `str` attributes
-    * `bool` attributes (using either the `--<arg-name>` or the `--<arg-name> <value>` syntax)
-    * `list` attributes
-    * `tuple` attributes
-* Parsing of multiple instances of a given dataclass, for the above-mentioned attribute types
-* Nested parsing of instances (dataclasses within dataclasses)
+
+- Parsing of attributes of built-in types:
+  - `int`, `float`, `str` attributes
+  - `bool` attributes (using either the `--<arg-name>` or the `--<arg-name> <value>` syntax)
+  - `list` attributes
+  - `tuple` attributes
+- Parsing of multiple instances of a given dataclass, for the above-mentioned attribute types
+- Nested parsing of instances (dataclasses within dataclasses)
 
 ## Possible Future Enhancements:
-* Parsing two different dataclasses which share a base class (this currently would cause a conflict for the base class arguments.
+
+- Parsing two different dataclasses which share a base class (this currently would cause a conflict for the base class arguments.
diff --git a/examples/ML/README.md b/examples/ML/README.md
@@ -1,6 +1,9 @@
 ## Use-Case Example: ML Scripts
+
 Let's look at a great use-case for `simple-parsing`: ugly ML code:
+
 ### Before:
+
 ```python
 import argparse
 
@@ -39,7 +42,9 @@ class MyModel():
 m = MyModel(data_dir, log_dir, checkpoint_dir, learning_rate, momentum)
 # Ok, what if we wanted to add a new hyperparameter?!
 ```
+
 ### After:
+
 ```python
 from dataclasses import dataclass
 from simple_parsing import ArgumentParser

diff --git a/examples/ML/ml_example_after.py b/examples/ML/ml_example_after.py
@@ -8,7 +8,7 @@
 
 @dataclass
 class MyModelHyperParameters:
-    """Hyperparameters of MyModel"""
+    """Hyperparameters of MyModel."""
 
     # Learning rate of the Adam optimizer.
     learning_rate: float = 0.05
@@ -18,7 +18,7 @@ class MyModelHyperParameters:
 
 @dataclass
 class TrainingConfig:
-    """Training configuration settings"""
+    """Training configuration settings."""
 
     data_dir: str = "/data"
     log_dir: str = "/logs"