Skip to content

Commit

Permalink
chore: define poetry exec targets for all tools (#695)
Browse files Browse the repository at this point in the history
  • Loading branch information
jnussbaum committed Dec 19, 2023
1 parent 70f6808 commit c04356b
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 21 deletions.
16 changes: 6 additions & 10 deletions .github/workflows/tests-on-push.yml
Expand Up @@ -39,7 +39,7 @@ jobs:
npm install -g markdown-link-validator
poetry exec check-links
- name: markdownlint
run: docker run -v $PWD:/workdir ghcr.io/igorshubovych/markdownlint-cli:latest --config .markdownlint.yml --ignore CHANGELOG.md "**/*.md"
run: poetry exec markdownlint

lint:
name: Lint code
Expand All @@ -64,14 +64,11 @@ jobs:
run: poetry install
# run tests
- name: Linting with ruff
run: >
poetry run ruff check .
--output-format=github
--ignore=A002,D101,D102,PLR0912,PLR0913,PLR0915,PLR2004,PLR5501,PLW0603
run: poetry exec ruff-check-github
- name: Formatting with ruff
run: poetry run ruff format .
run: poetry exec ruff-format
- name: Linting with mypy
run: poetry run mypy .
run: poetry exec mypy
- name: Linting with darglint
run: poetry exec darglint

Expand All @@ -98,7 +95,7 @@ jobs:
run: poetry install
# run tests
- name: unittests
run: poetry run pytest test/unittests/
run: poetry exec unittests

benchmarking:
name: Run Benchmarking
Expand Down Expand Up @@ -150,7 +147,6 @@ jobs:
# Install programs for local processing (fast xmlupload)
- name: Fast xmlupload - Install ffmpeg
uses: FedericoCarboni/setup-ffmpeg@v2

- name: Fast xmlupload - ImageMagick - look for cached AppImage
# If a cache is found that matches "key", the cached files are restored to "path".
# If no cache is found, this step is skipped, and when all jobs are done, the files in "path" are cached under "key".
Expand All @@ -166,6 +162,6 @@ jobs:
- name: start stack
run: poetry run dsp-tools start-stack --no-prune
- name: e2e tests
run: poetry run pytest test/e2e/
run: poetry exec e2e-tests
- name: stop stack # see if this command can run (it isn't tested anywhere else)
run: poetry run dsp-tools stop-stack
27 changes: 18 additions & 9 deletions README.md
Expand Up @@ -25,15 +25,15 @@ More details can be found in the [developers' documentation](https://docs.dasch.

## Quick Start

To get started quickly, without reading the details, just execute these commands
(a prior installation of [Homebrew](https://brew.sh/) is required):
To get started quickly, without reading the details, just execute these commands.

- `curl -sSL https://install.python-poetry.org | python3 -`
- `poetry self add poetry-exec-plugin`
- `poetry install`
- `poetry shell`
- `pre-commit install`
- `brew install imagemagick ffmpeg`
- `brew install imagemagick ffmpeg` (A prior installation of [Homebrew](https://brew.sh/) is required.)
- `npm install -g markdown-link-validator`

The remainder of this README explains these commands in more detail.

Expand Down Expand Up @@ -135,11 +135,11 @@ Please ensure you have only one pull request per feature.
## Testing

The tests of this repository
are written in the [unittest](https://docs.python.org/3/library/unittest.html) framework
and executed with the [pytest](https://docs.pytest.org) framework.
are partially written in the [unittest](https://docs.python.org/3/library/unittest.html) framework,
and partially in the [pytest](https://docs.pytest.org) framework.
There are two groups of tests:

- `test/unittests` they can be run directly,
- `test/unittests` can be run directly,
- `test/e2e` need a DSP stack running in the background.
A DSP stack can be started with the command
[`dsp-tools start-stack`](https://docs.dasch.swiss/latest/DSP-TOOLS/cli-commands/#start-stack)
Expand All @@ -164,6 +164,7 @@ When contributing to the project,
please make sure you use the same code style rules as we do.
We use the following linters:

- [markdown-link-validator](https://www.npmjs.com/package/markdown-link-validator) (no configuration)
- [MarkdownLint](https://github.com/igorshubovych/markdownlint-cli) (configured in `.markdownlint.yml`)
- [mypy](https://pypi.org/project/mypy/) (configured in `pyproject.toml`)
- [ruff](https://pypi.org/project/ruff/) (configured in `pyproject.toml`)
Expand All @@ -172,14 +173,22 @@ We use the following linters:
These linters are integrated in the GitHub CI pipeline,
so that every pull request is checked for code style violations.

Your code can be checked for style violations locally before they are committed:

- `poetry exec check-links`
- `poetry exec markdownlint`
- `poetry exec mypy`
- `poetry exec ruff-check`
- `poetry exec ruff-format`
- `poetry exec darglint`

In addition, there are [pre-commit hooks](#pre-commit-hooks)
that run Ruff and MarkdownLint locally before every commit.
This prevents you from committing code style violations.
Pre-commit is contained in the dependencies,
but before the first use, the hooks must be installed with `pre-commit install`
but before the first use, the hooks must be installed with `pre-commit install`.

Your code can be checked for style violations locally before they are committed.
Depending on your IDE the installation is as follows.
Depending on your IDE, there are extensions that emit warnings:


### VSCode
Expand Down
17 changes: 15 additions & 2 deletions pyproject.toml
Expand Up @@ -67,8 +67,7 @@ dsp-tools = "dsp_tools.cli.entry_point:main" # definition of the CLI entry poin
# plugin (https://github.com/keattang/poetry-exec-plugin) to define commands available for the developers,
# e.g. `poetry exec check-links`
check-links = """
markdown-link-validator \
./docs \
markdown-link-validator ./docs \
-i \\.\\/assets\\/.+ \
-i .+github\\.com\\/dasch\\-swiss\\/dsp-tools\\/settings \
-i .+github\\.com\\/dasch\\-swiss\\/ops-deploy\\/.+\
Expand All @@ -88,6 +87,20 @@ clean = """
rm -rf **/.mypy_cache
rm -rf *id2iri_mapping*.json
"""
markdownlint = """
docker run \
-v $PWD:/workdir ghcr.io/igorshubovych/markdownlint-cli:latest \
--config .markdownlint.yml \
--ignore CHANGELOG.md "**/*.md"
"""
ruff-check = "ruff check . --ignore=A002,D101,D102,PLR0912,PLR0913,PLR0915,PLR2004,PLR5501,PLW0603"
ruff-check-github = """
ruff check . --output-format=github --ignore=A002,D101,D102,PLR0912,PLR0913,PLR0915,PLR2004,PLR5501,PLW0603
"""
ruff-format = "ruff format ."
mypy = "mypy ."
unittests = "pytest test/unittests/"
e2e-tests = "pytest test/e2e/"


[build-system]
Expand Down

0 comments on commit c04356b

Please sign in to comment.