One step to speed up testing cycle based on code diff. For multiple languages.
Before: I have to run all the cases for even a tiny change, which takes 40 minutes. Which means 0.32 USD per push.
$ pytest --collect-only
============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-7.2.2, pluggy-0.12.0
rootdir: /Users/fengzhangchi/github_workspace/stagesepx
plugins: cov-2.7.1
collected 45 items
...
========================= 45 tests collected in 15.30s =========================After: Only need to run the influenced cases.
$ ./squ | xargs pytest --collect-only
============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-7.2.2, pluggy-0.12.0
plugins: cov-2.7.1
collected 5 items
...
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================= 5 tests collected in 14.93s ==========================All you need is a ./squ | xargs prefix.
You can find pre-build release binaries here.
One step on your command:
| Language | Before | After |
|---|---|---|
| Python | pytest |
squ | xargs pytest |
| Java | mvn test |
squ | xargs mvn test |
| Golang | go test |
squ | xargs go test |
Much benefits:
| Repo | Language | Analyze time | Before | After | Optimize |
|---|---|---|---|---|---|
| stagesepx (our own case) | Python | ~1 s | 45 | 0~15 | ~75% |
| Jacoco | Java | ~7 s | 1364 | 0~983 | ~50% |
| sibyl2 | Golang | ~1 s | 53 | 0~20 | ~70% |
Also, you can see our pipeline example if you want to make it work with github action.
This project inspired by Google.
Google used static analyze tech to scale down their time cost in test phase. In their case, only the influenced part will be evaluated and tested.
We implement it onto sibyl2. See Contribution for details.
squ --debug and you will see the log in squ-debug.log.
{"level":"info","ts":1678975693.395471,"caller":"upload/upload.go:178","msg":"upload batch: 0 - 50"}
{"level":"info","ts":1678975693.3977358,"caller":"indexer/indexer_base.go:78","msg":"indexer done"}
{"level":"info","ts":1678975693.400231,"caller":"UnitSqueezer/api.go:103","msg":"indexer ready"}
{"level":"info","ts":1678975693.412002,"caller":"extractor/extractor_git.go:56","msg":"version.go [3] => functions 0"}
{"level":"info","ts":1678975693.412069,"caller":"UnitSqueezer/api.go:113","msg":"diff calc ready: 0"}
{"level":"info","ts":1678975693.412087,"caller":"UnitSqueezer/api.go:120","msg":"case analyzer done, before: 3, after: 0"}
{"level":"info","ts":1678975695.459771,"caller":"UnitSqueezer/api.go:137","msg":"runner scope"}
{"level":"info","ts":1678975695.459826,"caller":"UnitSqueezer/api.go:139","msg":"no cases need to run"}
{"level":"info","ts":1678975695.459852,"caller":"UnitSqueezer/api.go:145","msg":"prepare stage finished, total cost: 2155 ms"}
{"level":"info","ts":1678975695.459863,"caller":"UnitSqueezer/api.go:147","msg":"runner cmd: --run=\"^$\""}
{"level":"info","ts":1678975695.460073,"caller":"server/app.go:113","msg":"sibyl server down: http: Server closed"}
squ --src ../SOMEWHERE
squ --before HEAD~5
Please use our issue board.
PR is always welcome.
This project consists of 4 parts.
- upload all the files to sibyl2
- search and tag all the test methods
- calc and tag all the test methods influencing scope
- calc diff between current and previous
- find methods influenced by diff
- search related cases
- build test commands for different frameworks
- call cmd