Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: define poetry exec targets for all tools #695

Merged
merged 2 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 6 additions & 10 deletions .github/workflows/tests-on-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
npm install -g markdown-link-validator
poetry exec check-links
- name: markdownlint
run: docker run -v $PWD:/workdir ghcr.io/igorshubovych/markdownlint-cli:latest --config .markdownlint.yml --ignore CHANGELOG.md "**/*.md"
run: poetry exec markdownlint

lint:
name: Lint code
Expand All @@ -64,14 +64,11 @@ jobs:
run: poetry install
# run tests
- name: Linting with ruff
run: >
poetry run ruff check .
--output-format=github
--ignore=A002,D101,D102,PLR0912,PLR0913,PLR0915,PLR2004,PLR5501,PLW0603
run: poetry exec ruff-check-github
- name: Formatting with ruff
run: poetry run ruff format .
run: poetry exec ruff-format
- name: Linting with mypy
run: poetry run mypy .
run: poetry exec mypy
- name: Linting with darglint
run: poetry exec darglint

Expand All @@ -98,7 +95,7 @@ jobs:
run: poetry install
# run tests
- name: unittests
run: poetry run pytest test/unittests/
run: poetry exec unittests

benchmarking:
name: Run Benchmarking
Expand Down Expand Up @@ -150,7 +147,6 @@ jobs:
# Install programs for local processing (fast xmlupload)
- name: Fast xmlupload - Install ffmpeg
uses: FedericoCarboni/setup-ffmpeg@v2

- name: Fast xmlupload - ImageMagick - look for cached AppImage
# If a cache is found that matches "key", the cached files are restored to "path".
# If no cache is found, this step is skipped, and when all jobs are done, the files in "path" are cached under "key".
Expand All @@ -166,6 +162,6 @@ jobs:
- name: start stack
run: poetry run dsp-tools start-stack --no-prune
- name: e2e tests
run: poetry run pytest test/e2e/
run: poetry exec e2e-tests
- name: stop stack # see if this command can run (it isn't tested anywhere else)
run: poetry run dsp-tools stop-stack
27 changes: 18 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ More details can be found in the [developers' documentation](https://docs.dasch.

## Quick Start

To get started quickly, without reading the details, just execute these commands
(a prior installation of [Homebrew](https://brew.sh/) is required):
To get started quickly, without reading the details, just execute these commands.

- `curl -sSL https://install.python-poetry.org | python3 -`
- `poetry self add poetry-exec-plugin`
- `poetry install`
- `poetry shell`
- `pre-commit install`
- `brew install imagemagick ffmpeg`
- `brew install imagemagick ffmpeg` (A prior installation of [Homebrew](https://brew.sh/) is required.)
- `npm install -g markdown-link-validator`

The remainder of this README explains these commands in more detail.

Expand Down Expand Up @@ -135,11 +135,11 @@ Please ensure you have only one pull request per feature.
## Testing

The tests of this repository
are written in the [unittest](https://docs.python.org/3/library/unittest.html) framework
and executed with the [pytest](https://docs.pytest.org) framework.
are partially written in the [unittest](https://docs.python.org/3/library/unittest.html) framework,
and partially in the [pytest](https://docs.pytest.org) framework.
There are two groups of tests:

- `test/unittests` they can be run directly,
- `test/unittests` can be run directly,
- `test/e2e` need a DSP stack running in the background.
A DSP stack can be started with the command
[`dsp-tools start-stack`](https://docs.dasch.swiss/latest/DSP-TOOLS/cli-commands/#start-stack)
Expand All @@ -164,6 +164,7 @@ When contributing to the project,
please make sure you use the same code style rules as we do.
We use the following linters:

- [markdown-link-validator](https://www.npmjs.com/package/markdown-link-validator) (no configuration)
- [MarkdownLint](https://github.com/igorshubovych/markdownlint-cli) (configured in `.markdownlint.yml`)
- [mypy](https://pypi.org/project/mypy/) (configured in `pyproject.toml`)
- [ruff](https://pypi.org/project/ruff/) (configured in `pyproject.toml`)
Expand All @@ -172,14 +173,22 @@ We use the following linters:
These linters are integrated in the GitHub CI pipeline,
so that every pull request is checked for code style violations.

Your code can be checked for style violations locally before they are committed:

- `poetry exec check-links`
- `poetry exec markdownlint`
- `poetry exec mypy`
- `poetry exec ruff-check`
- `poetry exec ruff-format`
- `poetry exec darglint`

In addition, there are [pre-commit hooks](#pre-commit-hooks)
that run Ruff and MarkdownLint locally before every commit.
This prevents you from committing code style violations.
Pre-commit is contained in the dependencies,
but before the first use, the hooks must be installed with `pre-commit install`
but before the first use, the hooks must be installed with `pre-commit install`.

Your code can be checked for style violations locally before they are committed.
Depending on your IDE the installation is as follows.
Depending on your IDE, there are extensions that emit warnings:


### VSCode
Expand Down
17 changes: 15 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,7 @@ dsp-tools = "dsp_tools.cli.entry_point:main" # definition of the CLI entry poin
# plugin (https://github.com/keattang/poetry-exec-plugin) to define commands available for the developers,
# e.g. `poetry exec check-links`
check-links = """
markdown-link-validator \
./docs \
markdown-link-validator ./docs \
-i \\.\\/assets\\/.+ \
-i .+github\\.com\\/dasch\\-swiss\\/dsp-tools\\/settings \
-i .+github\\.com\\/dasch\\-swiss\\/ops-deploy\\/.+\
Expand All @@ -88,6 +87,20 @@ clean = """
rm -rf **/.mypy_cache
rm -rf *id2iri_mapping*.json
"""
markdownlint = """
docker run \
-v $PWD:/workdir ghcr.io/igorshubovych/markdownlint-cli:latest \
--config .markdownlint.yml \
--ignore CHANGELOG.md "**/*.md"
"""
ruff-check = "ruff check . --ignore=A002,D101,D102,PLR0912,PLR0913,PLR0915,PLR2004,PLR5501,PLW0603"
ruff-check-github = """
ruff check . --output-format=github --ignore=A002,D101,D102,PLR0912,PLR0913,PLR0915,PLR2004,PLR5501,PLW0603
"""
ruff-format = "ruff format ."
mypy = "mypy ."
unittests = "pytest test/unittests/"
e2e-tests = "pytest test/e2e/"


[build-system]
Expand Down