Align naming of hatch scripts across our repos

[sjrl]

It would be great if we could align our script naming for hatch consistently across our different repos:

• https://github.com/deepset-ai/haystack
• https://github.com/deepset-ai/haystack-experimental
• https://github.com/deepset-ai/haystack-core-integrations

Currently we use variations on names like types vs typing and format vs fmt. Some common examples for each repo are:

• https://github.com/deepset-ai/haystack

hatch run test:format
hatch run test:types
hatch run test:lint

• https://github.com/deepset-ai/haystack-experimental

# format doesn't exist in experimental
hatch run test:typing
hatch run test:lint  # <-- Note: Does the same job as hatch run check + hatch run test:lint compared to main

• https://github.com/deepset-ai/haystack-core-integrations

hatch run lint:typing  # Note: uses different env name `lint` and `typing`
hatch run lint:style  # <-- We use black here instead of ruff format like our other repos
hatch run lint:fmt  # <-- similar to hatch run test:format from main

[srini047]

Hey @sjrl
I can take up this issue.

Will work on one repo at a time.

[srini047]

@sjrl
Hope this is a ideal [tool.hatch.envs.test.scripts]:

[tool.hatch.envs.test.scripts]
typing = "mypy --install-types --non-interactive {args:src tests}"
lint = [
  "ruff check {args:src tests}",
  "black --check --diff {args:src tests}",
]
format = [
  "black {args:src tests}",
  "ruff check --fix {args:src tests}",
]
all = ["typing", "lint"]

Open to suggestions and comments.

[srini047]

@sjrl Hope this is a ideal [tool.hatch.envs.test.scripts]:

[tool.hatch.envs.test.scripts]
typing = "mypy --install-types --non-interactive {args:src tests}"
lint = [
"ruff check {args:src tests}",
"black --check --diff {args:src tests}",
]
format = [
"black {args:src tests}",
"ruff check --fix {args:src tests}",
]
all = ["typing", "lint"]
Open to suggestions and comments.

Or a more comprehensive like this:

[tool.hatch.envs.test.scripts]
typing = "mypy --install-types --non-interactive --explicit-package-bases {args:src/}"
lint = [
    "ruff check {args:.}",
    "pylint -ry -j 0 {args:src/}",
]
format = "ruff format {args:.}"
format-check = "ruff format --check {args:.}"
fix = "ruff check --fix {args:.}"
test = "pytest {args}"
all = ["typing", "lint", "format-check", "test"]

[anakin87]

When adapting the commands for core-integrations, we should take deepset-ai/haystack-core-integrations#1771 (comment) into consideration.

[anakin87]

I'd like to investigate this.

Haystack

[tool.hatch.envs.default.scripts]
release-note = "reno new {args}"
check = "ruff check {args}"
fix = "ruff check --fix"
format = "ruff format {args}"
format-check = "ruff format --check {args}"

[tool.hatch.envs.test.scripts]
unit = 'pytest --cov-report xml:coverage.xml --cov="haystack" -m "not integration" {args:test}'
integration = 'pytest --maxfail=5 -m "integration" {args:test}'
integration-only-fast = 'pytest --maxfail=5 -m "integration and not slow" {args:test}'
integration-only-slow = 'pytest --maxfail=5 -m "integration and slow" {args:test}'

types = "mypy --install-types --non-interactive --cache-dir=.mypy_cache/ {args:haystack}"
lint = "pylint -ry -j 0 {args:haystack}"

we also use precommit

---

experimental

[tool.hatch.envs.test.scripts]
unit = 'pytest --cov-report xml:coverage.xml --cov="haystack_experimental" -m "not integration" {args:test}'
integration = 'pytest --reruns 3 --reruns-delay 60 -x --maxfail=5 -m "integration and not unstable" {args:test}'
typing = "mypy --install-types --non-interactive {args:haystack_experimental}"
lint = [
  "ruff check {args:haystack_experimental}",
  "pylint -ry -j 0 {args:haystack_experimental}",
]
test-cov = "coverage run -m pytest -m \"not unstable\" {args:test}"
cov-report = ["- coverage combine", "coverage xml"]
cov = ["test-cov", "cov-report"]

---

core integrations

[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "coverage run -m pytest {args:tests}"
test-cov-retry = "test-cov --reruns 3 --reruns-delay 30 -x"
cov-report = ["- coverage combine", "coverage report"]
cov = ["test-cov", "cov-report"]
cov-retry = ["test-cov-retry", "cov-report"]
docs = ["pydoc-markdown pydoc/config.yml"]

[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive --explicit-package-bases {args:src/ tests}"
style = ["ruff check {args:.}", "black --check --diff {args:.}"]
fmt = ["black {args:.}", "ruff check --fix {args:.}", "style"]
all = ["style", "typing"]

[anakin87]

I would propose to align and simplify our commands, as follows.

native command hatch fmt for format + ruff linting (checking + fixing if possible)

hatch run test:unit for unit tests
hatch run test:integration for integration tests
hatch run test:all for all tests

hatch run test:types for mypy/type checking
hatch run test:lint for pylint

Note: MyPy and Pylint require project dependencies to be installed and do not work correctly in detached environments. (deepset-ai/haystack-core-integrations#1771 (comment)).

These developer-facing commands should be standardized across our repositories. Each repo may define additional utility commands as needed, primarily used for CI.

@sjrl WDYT?

[sjrl]

@anakin87 that sounds good to me!

Also maybe we could reduce the cov related commands? Do we need so many different ones in each of the repos? E.g. test-cov, test-cov-retry, cov-report, cov, and cov-retry?

[anakin87]

Also maybe we could reduce the cov related commands? Do we need so many different ones in each of the repos? E.g. test-cov, test-cov-retry, cov-report, cov, and cov-retry?

This suggestion seems to make sense.

[anakin87]

After some exploration, I changed my mind about hatch fmt:

• it would tie us to Hatch
• does not play well ruff pre-commit hook

For this reason, I would introduce hatch run fmt instead, which would do the same but we would have more granular control on it.