chore: setup turbo

[sripwoud]

This PR setups turbo as task handler for the following tasks: formatting, linting, building and testing.
It is not as complex as the proofcarryingdata/zupass#1590 referenced in #207.
But it already speeds up considerably task execution in local environments, and speeds it up somewhat in the ci environments (turbo cache isn't setup yet in ci, but turbo executes all tasks in parallel instead of previously sequential && chaining).

Description

This is only a devops/devtool change.

Before	After
none of the tasks related to building, formatting, linting or testing are cached.	tasks related to building, formatting, linting or testing are cached.
yarn test:libraries	yarn turbo test:libraries

Test plan

Check caching

1. Run one of the tasks related to formatting, linting, building and testing.
2. Run it again: cache should be hit.
3. Modify some files listed in the corresponding inputs.
4. Run it again: cache should be missed

Check tasks dependencies

Now all the scripts (especially the tests and linting ones) should execute successfully on a fresh repo without requiring to run any build scripts manually beforehand.
ex:

for dir in artifacts dist typechain-types node_modules;do find . -type d -name "$dir" -exec rm -rf {} +;done && yarn && yarn test --force

for dir in artifacts dist typechain-types node_modules;do find . -type d -name "$dir" -exec rm -rf {} +;done && yarn && yarn style --force

Related Issue(s)

Closes (fully or partially?) #207

Other information

A combination of root level (//#task) and package level (task) tasks are defined in turbo.json.
As the turbo.json defines custom inputs for all tasks, we loose the default turbo behavior of ignoring gitignored files, and we need to explicitly define ignore globs for irrelevant files.
As this monorepo is quite big and involves lot of config/build/log/cache artifacts, I am not sure the inputs globs I defined are 100% correct.

What this PR does not do
Optimize the GH worklfows (would require setting .turbo cache in the GH actions environment).
I suggest doing this in a separate PR later.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have run yarn style without getting any errors
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[cedoor]

@sripwoud wow thank you so much for this!

Looks like the style job keeps executing indefinitely, isn't it?

[sripwoud]

@sripwoud wow thank you so much for this!

Looks like the style job keeps executing indefinitely, isn't it?

completes successfully after retriggering it

[cedoor]

LGTM :)

[sripwoud]

hmm style workflow is taking forever again. Cancelling and re start fixes it. But something is wrong.
Not sure why (concurrent jobs, turbo cache not being shared or turbo locking some files?). Will need to debug

[sripwoud]

hmm style workflow is taking forever again. Cancelling and re start fixes it. But something is wrong. Not sure why (concurrent jobs, turbo cache not being shared or turbo locking some files?). Will need to debug

@cedoor fixed
Problem was wrong ordering of tasks in dependsOn values, not excluding some output files from inputs, trying to build some things at the same time...

I updated the test plan to cover checking the tasks dependencies are well defined.

[cedoor]

hmm style workflow is taking forever again. Cancelling and re start fixes it. But something is wrong. Not sure why (concurrent jobs, turbo cache not being shared or turbo locking some files?). Will need to debug

@cedoor fixed Problem was wrong ordering of tasks in dependsOn values, not excluding some output files from inputs, trying to build some things at the same time...

I updated the test plan to cover checking the tasks dependencies are well defined.

Great! It's ready then 👍🏽

[gitpoap-bot]

Congrats, your important contribution to this open-source project has earned you a GitPOAP!

GitPOAP: 2024 ZK-KIT Contributor:

Head to gitpoap.io & connect your GitHub account to mint!

Learn more about GitPOAPs here.