Run blackbox tests in CI #260
Conversation
| @@ -28,6 +28,47 @@ jobs: | |||
| run: make pip | |||
| - name: Build and Test | |||
| run: make build-and-test | |||
There was a problem hiding this comment.
I'm mostly out of ideas here.
Locally via https://github.com/nektos/act, it works at these commits:
It's unclear to me how to otherwise debug a failed invocation (e.g. https://github.com/algorand/pyteal/runs/5664812490?check_suite_focus=true). It's quite possible I'm taking an approach that's a poor fit.
There was a problem hiding this comment.
Thanks for suggesting act. Really useful!!!
| - name: Install required os level applications | ||
| run: | | ||
| sudo apt update -y | ||
| sudo apt install -y curl git nodejs python-is-python3 python3-pip |
There was a problem hiding this comment.
As part of bringing these changes to a mergeable state, I think we ought to disambiguate steps needed for act.
- In order to get act to pass, I sourced various dependencies that I believe are not needed for github.
- Disambiguation here means creating 1 or more steps dedicated to act dependencies. Goal is to make it easier to maintain the build.
- I think the following dependencies are act-specific though it requires a round of testing:
curl,nodejs,python3-pip- Docker install (e.g.
docker-ce) - Setup docker compose
|
Bringing a comment over from #249 that's better placed here:
|
From my understanding the present solution is an optimized version of 3.: A simple sandbox is downloaded but it's cached so -in practice- it's almost as good as 2. Also, we see from experience that it's taking around 5 minutes to run the tests. It's slower than the 1 minute times we had before the test, but not nearly as bad as the wait times on the SDK's and on |
|
Closing - We've evolved these techniques via algorand/graviton#1. #249 will directly bring in the CI test running. |
* Linting related changes:
*`.flake8`:
* Based on my experience in `graviton` and a couple of extra ignores for formatting that was conflicting with `black`
* replace `from pyteal import *` with `import pyteal as pt` in our tests, and for other files ignore the * import error on a per file basis (_this is the reason line changes number in the thousands which amounts to almost 25% of the entire codebase_)
* Incorporate #277
* Additional changes:
* Removing `requirements.txt` (but keeping `docs/requirements.txt`)
* `setup.py`: `extras_require={"development" ... ` replaces the former `requirements.txt`
* `.github/workflows/build.yml` + `Makefile`: unify as much of the logic as possible. Needed regular `python` instead of `python-slim` in order to have `make`. `black` is applied to all python files, as before, and `flake8` is as well. `mypy` is applied to `scripts` in addition to `pyteal` (in upcoming #260 this will be expanded to `tests` as well).
* `README.md` - applying [mardkownlint](https://github.com/DavidAnson/markdownlint) (without automation)
* Linting related changes:
*`.flake8`:
* Based on my experience in `graviton` and a couple of extra ignores for formatting that was conflicting with `black`
* replace `from pyteal import *` with `import pyteal as pt` in our tests, and for other files ignore the * import error on a per file basis (_this is the reason line changes number in the thousands which amounts to almost 25% of the entire codebase_)
* Incorporate algorand#277
* Additional changes:
* Removing `requirements.txt` (but keeping `docs/requirements.txt`)
* `setup.py`: `extras_require={"development" ... ` replaces the former `requirements.txt`
* `.github/workflows/build.yml` + `Makefile`: unify as much of the logic as possible. Needed regular `python` instead of `python-slim` in order to have `make`. `black` is applied to all python files, as before, and `flake8` is as well. `mypy` is applied to `scripts` in addition to `pyteal` (in upcoming algorand#260 this will be expanded to `tests` as well).
* `README.md` - applying [mardkownlint](https://github.com/DavidAnson/markdownlint) (without automation)
Defines
Makefilefacilities to run blackbox tests from #249 as part of the build.