Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make it easier to run a subset of the buildpack unit tests locally #1020

Closed
edmorley opened this issue Jul 28, 2020 · 0 comments · Fixed by #1021
Closed

Make it easier to run a subset of the buildpack unit tests locally #1020

edmorley opened this issue Jul 28, 2020 · 0 comments · Fixed by #1021
Assignees
@edmorley

Comments

@edmorley
Copy link
Member

edmorley commented Jul 28, 2020 •
edited

The buildpack's unit tests are split into three: tests/deps, tests/features and tests/versions.

On travis these are run separately, via hardcoded invocations in .travis.yml.

However locally there is currently no make target to allow doing this - instead all tests for a stack have to be run using eg make test-heroku-18 - so on several occasions I've hand-edited Makefile locally to make the target only run one of the three test suites to speed up testing.

This is a pain since I have to revert the changes each time -- we should instead add support for doing this natively.
@edmorley edmorley self-assigned this Jul 28, 2020
@edmorley edmorley mentioned this issue Jul 28, 2020
Merged
edmorley added a commit that referenced this issue Jul 28, 2020
@edmorley
Tests: Support running only a subset of the test suites
6c91739 
Before `make test` ran all unit test suites against all stacks, which
would take up to an hour locally. This could be sped up by using one of
the stack-specific targets (such as `make test-heroku-18`), however
there was still no way to only run one of the test suites.

Now `make test` can be controlled more precisely using optional `STACK`
and `TEST_CMD` arguments, eg:

`make test STACK=heroku-16 TEST_CMD=test/versions`

Travis has now been made to use this feature, which unblocks future
Travis speedups (such as splitting the jobs up further in #1018) and
means on Travis the correct Docker image is now used.

The `tests.sh` script has been removed since it's unused after #839 and
redundant given the make targets.

Fixes #958.
Fixes #1020.
edmorley added a commit that referenced this issue Jul 28, 2020
@edmorley
Tests: Support running only a subset of the test suites (#1021)
93a5b40 
Previously `make test` ran all unit test suites against all stacks, which
would take up to an hour locally. This could be sped up by using one of
the stack-specific targets (such as `make test-heroku-18`), however
there was still no way to only run one of the test suites.

Now `make test` can be controlled more precisely using optional `STACK`
and `TEST_CMD` arguments, eg:

`make test STACK=heroku-16 TEST_CMD=test/versions`

Travis has now been made to use this feature, which unblocks future
Travis speedups (such as splitting the jobs up further in #1018) and
means on Travis the correct Docker image is now used (see #958).

The `tests.sh` script has been removed since it's unused after #839 and
redundant given the make targets.

Fixes #958.
Fixes #1020.
dryan pushed a commit to dryan/heroku-buildpack-python that referenced this issue Nov 19, 2020
@edmorley @dryan
Tests: Support running only a subset of the test suites (heroku#1021)
5582bb1 
Previously `make test` ran all unit test suites against all stacks, which
would take up to an hour locally. This could be sped up by using one of
the stack-specific targets (such as `make test-heroku-18`), however
there was still no way to only run one of the test suites.

Now `make test` can be controlled more precisely using optional `STACK`
and `TEST_CMD` arguments, eg:

`make test STACK=heroku-16 TEST_CMD=test/versions`

Travis has now been made to use this feature, which unblocks future
Travis speedups (such as splitting the jobs up further in heroku#1018) and
means on Travis the correct Docker image is now used (see heroku#958).

The `tests.sh` script has been removed since it's unused after heroku#839 and
redundant given the make targets.

Fixes heroku#958.
Fixes heroku#1020.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@edmorley edmorley

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Tests: Support running only a subset of the test suites heroku/heroku-buildpack-python
1 participant
@edmorley