Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): Cache yarn and playwright dependencies in CI #2697

Merged
merged 33 commits into from
Sep 15, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
4c04d21
chore: remove unnecessary gitignores
TomAFrench Sep 14, 2023
6d89ee6
chore(ci): cache yarn dependencies
TomAFrench Sep 14, 2023
d428d33
chore(ci): cache playwright
TomAFrench Sep 14, 2023
0919c63
chore: fix tests on mac publishing
TomAFrench Sep 14, 2023
a2c153b
chore: add `noir_wasm_testing.json` to gitignore
TomAFrench Sep 14, 2023
10b147d
chore: separate building from testing `noirc_abi_wasm`
TomAFrench Sep 14, 2023
aa4f1c9
chore: remove unnecessary `working-directory`
TomAFrench Sep 14, 2023
4e37407
chore: remove unwanted `working-directory`s
TomAFrench Sep 14, 2023
e8418af
chore: add descriptions to github actions
TomAFrench Sep 14, 2023
06d052a
chore: add step to poke yarn
TomAFrench Sep 14, 2023
8c7c51c
chore: run node and browser tests separately
TomAFrench Sep 14, 2023
f9c1da1
chore: use consistent set of browsers across tests
TomAFrench Sep 14, 2023
f73ae2f
chore: remove explicit node from setup action
TomAFrench Sep 14, 2023
e8ed744
chore: update dependencies
TomAFrench Sep 14, 2023
9d9c947
chore: split build and tests jobs in `abi_wasm.yml`
TomAFrench Sep 15, 2023
a416626
Update .github/workflows/abi_wasm.yml
kevaundray Sep 15, 2023
f7a86ee
chore: refresh lockfile
TomAFrench Sep 15, 2023
5aff7f6
chore: remove `packageManager" fields from workspace packages
TomAFrench Sep 15, 2023
91379fd
chore: clean up straggler `packageManager`
TomAFrench Sep 15, 2023
dbaf9de
chore: pass full `package.json` into build artifact
TomAFrench Sep 15, 2023
10ed648
chore(ci): fix playwright caching
TomAFrench Sep 15, 2023
f344d75
chore: nits
TomAFrench Sep 15, 2023
ac801ee
chore: fix version querying
TomAFrench Sep 15, 2023
9647178
chore: remove reference to `result` directory
TomAFrench Sep 15, 2023
88ec481
Merge branch 'master' into tf/split-build-and-test
TomAFrench Sep 15, 2023
376499a
Merge branch 'tf/split-build-and-test' into tf/yarn-caching
TomAFrench Sep 15, 2023
4e974cf
chore: fix merge
TomAFrench Sep 15, 2023
33675dd
chore: fix merge
TomAFrench Sep 15, 2023
6f4cc8f
chore: undo changes to playwright cache key
TomAFrench Sep 15, 2023
b465812
chore: remove yarn poking
TomAFrench Sep 15, 2023
cd82505
Merge branch 'master' into tf/yarn-caching
TomAFrench Sep 15, 2023
05ead8b
chore: revert changes to browser tests
TomAFrench Sep 15, 2023
b7b4be7
chore: disable extra browsers again
TomAFrench Sep 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/actions/install-playwright/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Install Playwright
description: Installs Playwright and its dependencies and caches them.

runs:
using: composite
steps:
- name: Query playwright version
shell: bash
run: echo "PLAYWRIGHT_VERSION=$(yarn workspace @noir-lang/noirc_abi info @web/test-runner-playwright --json | jq .children.Version | tr -d '"')" >> $GITHUB_ENV

- name: Cache playwright binaries
uses: actions/cache@v3
id: playwright-cache
with:
path: |
~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}

- name: Install playwright deps
shell: bash
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: |
npx playwright install
npx playwright install-deps
16 changes: 3 additions & 13 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -1,26 +1,16 @@
name: Setup

inputs:
working-directory:
default: ./
required: false
name: Install Yarn dependencies
description: Installs the workspace's yarn dependencies and caches them

runs:
using: composite
steps:
- name: Install node
uses: actions/setup-node@v3
with:
node-version: 18.17.1
- name: Cache
Comment on lines -11 to -14
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You really should be using setup-node to have specific versions of node instead of the outdated one that ships with actions.

Additionally, the setup-node action has the yarn caching directly built-in.

uses: actions/cache@v3
id: cache
with:
path: "**/node_modules"
key: yarn-v1-${{ hashFiles('**/yarn.lock') }}
- name: Install
run: |
cd ${{ inputs.working-directory }}
yarn --immutable
run: yarn --immutable
shell: bash
if: steps.cache.outputs.cache-hit != 'true'
46 changes: 20 additions & 26 deletions .github/workflows/abi_wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ concurrency:
cancel-in-progress: true

jobs:
noirc-abi-wasm-build:
build:
runs-on: ubuntu-latest
env:
CACHED_PATH: /tmp/nix-cache
Expand Down Expand Up @@ -72,35 +72,29 @@ jobs:
path: ${{ env.UPLOAD_PATH }}
retention-days: 10

- name: Install workspace dependencies
run: |
yarn install --immutable

- name: Run node tests
run: |
yarn workspace @noir-lang/noirc_abi test
test:
runs-on: ubuntu-latest
needs: [build]
steps:
- name: Checkout sources
uses: actions/checkout@v3

- name: Cache playwright binaries
uses: actions/cache@v3
id: playwright-cache
- name: Download wasm package artifact
uses: actions/download-artifact@v3
with:
path: |
~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}
name: noirc_abi_wasm
path: ./tooling/noirc_abi_wasm

- name: Install Yarn dependencies
uses: ./.github/actions/setup

- name: Query playwright version
working-directory: ./tooling/noirc_abi_wasm
run: echo "PLAYWRIGHT_VERSION=$(yarn info @web/test-runner-playwright --json | jq .children.Version)" >> $GITHUB_ENV
- name: Run node tests
run: yarn workspace @noir-lang/noirc_abi test

- name: Install playwright deps
if: steps.playwright-cache.outputs.cache-hit != 'true'
working-directory: ./tooling/noirc_abi_wasm
run: |
npx playwright install
npx playwright install-deps
- name: Install Playwright
uses: ./.github/actions/install-playwright

- name: Run browser tests
working-directory: ./tooling/noirc_abi_wasm
run: |
yarn workspace @noir-lang/noirc_abi test:browser
run: yarn workspace @noir-lang/noirc_abi test:browser

10 changes: 8 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,14 @@ jobs:
path: ./dist/*
retention-days: 3

- name: Install Yarn dependencies
if: matrix.target == 'x86_64-apple-darwin'
uses: ./.github/actions/setup

- name: Test built artifact
if: matrix.target == 'x86_64-apple-darwin'
run: |
cp ./target/${{ matrix.target }}/release/nargo ~/.cargo/bin/
yarn install --immutable
yarn workspace release-tests test

- name: Upload binaries to release tag
Expand Down Expand Up @@ -139,11 +142,14 @@ jobs:
path: ./dist/*
retention-days: 3

- name: Install Yarn dependencies
if: startsWith(matrix.target, 'x86_64-unknown-linux')
uses: ./.github/actions/setup

- name: Test built artifact
if: startsWith(matrix.target, 'x86_64-unknown-linux')
run: |
cp ./target/${{ matrix.target }}/release/nargo ~/.cargo/bin/
yarn install --immutable
yarn workspace release-tests test

- name: Upload binaries to release tag
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/test-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,11 @@ jobs:
path: ${{ env.UPLOAD_PATH }}
retention-days: 3

- name: Install Yarn dependencies
uses: ./.github/actions/setup

- name: Setup `integration-tests`
run: |
yarn install --immutable
yarn workspace @noir-lang/noir-source-resolver build

- name: Run `integration-tests`
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test-source-resolver.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ jobs:
- name: Checkout
uses: actions/checkout@v3

- name: Install dependencies
run: yarn install --immutable
- name: Install Yarn dependencies
uses: ./.github/actions/setup

- name: Build @noir-lang/source-resolver
run: yarn workspace @noir-lang/source-resolver build
Expand Down
18 changes: 11 additions & 7 deletions .github/workflows/wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,16 +132,20 @@ jobs:
chmod +x $nargo_binary
$nargo_binary compile

- name: Install Yarn dependencies
uses: ./.github/actions/setup

- name: Install Playwright
uses: ./.github/actions/install-playwright

- name: Install dependencies
run: |
npx playwright install
npx playwright install-deps
cp -r ./compiler/wasm/downloaded/nodejs ./compiler/wasm
cp -r ./compiler/wasm/downloaded/web ./compiler/wasm
yarn install --immutable
yarn workspace @noir-lang/source-resolver build

- name: Run tests
run: |
yarn workspace @noir-lang/noir_wasm test:browser
yarn workspace @noir-lang/noir_wasm test:node
- name: Run node tests
run: yarn workspace @noir-lang/noir_wasm test:node

- name: Run browser tests
run: yarn workspace @noir-lang/noir_wasm test:browser
4 changes: 3 additions & 1 deletion compiler/integration-tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
"@noir-lang/source-resolver": "workspace:*",
"@web/dev-server-esbuild": "^0.3.6",
"@web/test-runner": "^0.15.3",
"@web/test-runner-webdriver": "^0.7.0"
"@web/test-runner-webdriver": "^0.7.0",
"fflate": "^0.8.0",
"smol-toml": "^1.1.2"
}
}
9 changes: 1 addition & 8 deletions compiler/wasm/.gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1 @@
# Yarn
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
noir-script/target
6 changes: 1 addition & 5 deletions compiler/wasm/installPhase.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ export self_path=$(dirname "$(readlink -f "$0")")

mkdir -p $out
cp $self_path/README.md $out/
cp $self_path/package.json $out/
cp -r $self_path/nodejs $out/
cp -r $self_path/web $out/

# The main package.json contains several keys which are incorrect/unwanted when distributing.
cat $self_path/package.json \
| jq 'del(.private, .devDependencies, .scripts, .packageManager)' \
> $out/package.json
1 change: 0 additions & 1 deletion compiler/wasm/noir-script/target/noir_wasm_testing.json

This file was deleted.

2 changes: 1 addition & 1 deletion compiler/wasm/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
"package.json"
],
"sideEffects": false,
"packageManager": "yarn@3.5.1",
"repository": {
"type": "git",
"url": "https://github.com/noir-lang/noir.git"
Expand All @@ -29,6 +28,7 @@
"@noir-lang/source-resolver": "workspace:*"
},
"devDependencies": {
"@esm-bundle/chai": "^4.3.4-fix.0",
"@web/dev-server-esbuild": "^0.3.6",
"@web/test-runner": "^0.15.3",
"@web/test-runner-playwright": "^0.10.0",
Expand Down
3 changes: 0 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,13 @@
"test:integration": "yarn workspace integration-tests test"
},
"devDependencies": {
"@esm-bundle/chai": "^4.3.4-fix.0",
"@typescript-eslint/eslint-plugin": "^5.59.5",
"@typescript-eslint/parser": "^5.59.5",
"chai": "^4.3.7",
"eslint": "^8.40.0",
"eslint-plugin-prettier": "^4.2.1",
"fflate": "^0.8.0",
"mocha": "^10.2.0",
"prettier": "^2.8.8",
"smol-toml": "^1.1.2",
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
},
Expand Down
8 changes: 0 additions & 8 deletions tooling/noirc_abi_wasm/.gitignore

This file was deleted.

6 changes: 1 addition & 5 deletions tooling/noirc_abi_wasm/installPhase.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ export self_path=$(dirname "$(readlink -f "$0")")

mkdir -p $out
cp $self_path/README.md $out/
cp $self_path/package.json $out/
cp -r $self_path/nodejs $out/
cp -r $self_path/web $out/

# The main package.json contains several keys which are incorrect/unwanted when distributing.
cat $self_path/package.json \
| jq 'del(.private, .devDependencies, .scripts, .packageManager)' \
> $out/package.json
2 changes: 1 addition & 1 deletion tooling/noirc_abi_wasm/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@
"type": "git",
"url": "https://github.com/noir-lang/noir.git"
},
"packageManager": "yarn@3.5.1",
"scripts": {
"build": "bash ./build.sh",
"test": "env TS_NODE_COMPILER_OPTIONS='{\"module\": \"commonjs\" }' mocha",
"test:browser": "web-test-runner",
"lint": "NODE_NO_WARNINGS=1 eslint . --ext .ts --ignore-path ./.eslintignore --max-warnings 0"
},
"devDependencies": {
"@esm-bundle/chai": "^4.3.4-fix.0",
"@web/dev-server-esbuild": "^0.3.6",
"@web/test-runner": "^0.15.3",
"@web/test-runner-playwright": "^0.10.0",
Expand Down
9 changes: 6 additions & 3 deletions tooling/noirc_abi_wasm/test/browser/abi_encode.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
import { expect } from "@esm-bundle/chai";
import initNoirAbi, { abiEncode, abiDecode, WitnessMap } from "@noir-lang/noirc_abi";
import initNoirAbi, {
abiEncode,
abiDecode,
WitnessMap,
} from "@noir-lang/noirc_abi";
import { DecodedInputs } from "../types";

beforeEach(async () => {
Expand All @@ -12,7 +16,6 @@ it("recovers original inputs when abi encoding and decoding", async () => {
const initial_witness: WitnessMap = abiEncode(abi, inputs, null);
const decoded_inputs: DecodedInputs = abiDecode(abi, initial_witness);


expect(BigInt(decoded_inputs.inputs.foo)).to.be.equal(BigInt(inputs.foo));
expect(BigInt(decoded_inputs.inputs.bar[0])).to.be.equal(
BigInt(inputs.bar[0])
Expand All @@ -21,4 +24,4 @@ it("recovers original inputs when abi encoding and decoding", async () => {
BigInt(inputs.bar[1])
);
expect(decoded_inputs.return_value).to.be.null;
});
});
Loading
Loading