Tracking issue: --persist-doctests flag for rustdoc #56925

QuietMisdreavus · 2018-12-17T19:48:11Z

First implemented in #56189, this flag allows users to save the compiled binaries created when rustdoc runs documentation tests.

Pending work:

Land rustdoc: Add option to persist doc test executables #56189
Change the folder naming scheme? i.e. rustdoc: Add option to persist doc test executables #56189 (comment)
Related option to also save the processed code, for debugging purposes?
Ability to cross-compile doctests and either save the binaries or run them through some test runner
Combining the binaries into a singular test runner, like how libtest saves a single binary that runs unit tests

jonhoo · 2020-02-10T21:02:43Z

This would be really handy to have for getting code coverage for doctests. @QuietMisdreavus do you know what is left of the process to land this? Even in its current form it is very useful.

tarpaulin needs rust-lang/rust#56925 to get coverage for doctests, which is only available on nightly.

Use of nightly compiler is required due to unstable feature <rust-lang/rust#56925>.

This replaces tarpaulin with llvm-cov which seems to be much faster, works across platforms, and should be more reliable moving forward since it uses rustc native coverage information. Use of nightly compiler is required due to unstable features <rust-lang/rust#84605> and <rust-lang/rust#56925>.

…caccia` Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "focaccia-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov focaccia*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov focaccia*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-focaccia-s3-backup-20220820215201567600000003 role-session-name: GitHubActionsRustCodeCoverage@focaccia - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/focaccia/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/focaccia/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/focaccia/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/focaccia/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/focaccia/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "boba-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov boba*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov boba*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-boba-s3-backup-2022082021520156830000000a role-session-name: GitHubActionsRustCodeCoverage@boba - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/boba/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/boba/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/boba/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/boba/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/boba/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

…rftime-ruby` Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "strftime-ruby-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov strftime-ruby*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov strftime-ruby*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-strftime-ruby-s3-backup-20220817011212567800000002 role-session-name: GitHubActionsRustCodeCoverage@strftime-ruby - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/strftime-ruby/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/strftime-ruby/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/strftime-ruby/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/strftime-ruby/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/strftime-ruby/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "roe-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov roe*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov roe*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-roe-s3-backup-20220820215201568000000007 role-session-name: GitHubActionsRustCodeCoverage@roe - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/roe/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/roe/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/roe/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/roe/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/roe/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

…w-parts` Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "raw-parts-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov raw-parts*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov raw-parts*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-raw-parts-s3-backup-20220820215201567700000005 role-session-name: GitHubActionsRustCodeCoverage@raw-parts - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/raw-parts/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/raw-parts/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/raw-parts/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/raw-parts/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/raw-parts/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

…taglio` Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "intaglio-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov intaglio*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov intaglio*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-intaglio-s3-backup-20220820215201567700000006 role-session-name: GitHubActionsRustCodeCoverage@intaglio - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/intaglio/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/intaglio/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/intaglio/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/intaglio/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/intaglio/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

…six-space` Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "posix-space-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov posix-space*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov posix-space*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-posix-space-s3-backup-20220820215201568100000009 role-session-name: GitHubActionsRustCodeCoverage@posix-space - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/posix-space/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/posix-space/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/posix-space/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/posix-space/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/posix-space/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```

…nd_mt` Managed by Terraform. ## Contents ``` --- name: Code Coverage "on": push: branches: - trunk pull_request: branches: - trunk jobs: generate: name: Generate permissions: id-token: write contents: read runs-on: ubuntu-latest env: RUST_BACKTRACE: 1 CARGO_NET_GIT_FETCH_WITH_CLI: true steps: - name: Checkout repository uses: actions/checkout@v3 - name: Install Rust toolchain uses: actions-rs/toolchain@v1 with: toolchain: nightly profile: minimal override: true components: llvm-tools-preview - name: Setup grcov run: | release_url="$(curl \ -H "Accept: application/vnd.github.v3+json" \ https://api.github.com/repos/mozilla/grcov/releases | \ jq -r '.[0].assets | map(select(.browser_download_url | test(".*x86_64-unknown-linux-musl.tar.bz2$"))) | .[0].browser_download_url')" curl -sL "$release_url" | sudo tar xvj -C /usr/local/bin/ - name: Generate coverage env: LLVM_PROFILE_FILE: "rand_mt-%m.profraw" RUSTFLAGS: "-C instrument-coverage" # Unstable feature: rust-lang/rust#56925 RUSTDOCFLAGS: "-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests" run: | cargo +nightly test --lib cargo +nightly test --doc - name: Generate HTML report run: grcov rand_mt*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage - name: Generate detailed JSON report run: grcov rand_mt*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage/coverage.json - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@master if: github.ref == 'refs/heads/trunk' with: aws-region: us-west-2 role-to-assume: arn:aws:iam::447522982029:role/gha-rand_mt-s3-backup-20220820215201568000000008 role-session-name: GitHubActionsRustCodeCoverage@rand_mt - name: Show AWS caller identity if: github.ref == 'refs/heads/trunk' run: aws sts get-caller-identity - name: Upload archives to S3 if: github.ref == 'refs/heads/trunk' run: | aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/rand_mt/ --delete --sse AES256 --exclude '*' --include '*.svg' --content-type 'image/svg+xml' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/rand_mt/ --delete --sse AES256 --exclude '*' --include '*.html' --content-type 'text/html' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/rand_mt/ --delete --sse AES256 --exclude '*' --include '*.json' --content-type 'application/json' aws s3 sync target/coverage/ s3://artichoke-forge-code-coverage-us-west-2/rand_mt/ --delete --sse AES256 --include '*' --exclude '*.svg' --exclude '*.html' --exclude '*.json' - name: Check missed lines run: | curl -s https://codecov.artichokeruby.org/rand_mt/coverage.json | python -c '\ import sys, json; \ \ trunk_coverage = json.loads(sys.stdin.read()); \ print("On trunk: "); \ print("coveragePercent =", trunk_coverage["coveragePercent"]); \ print("linesCovered =", trunk_coverage["linesCovered"]); \ print("linesMissed =", trunk_coverage["linesMissed"]); \ print("linesTotal =", trunk_coverage["linesTotal"]); \ print(""); \ \ branch_coverage = json.load(open("target/coverage/coverage.json")) print("On PR branch: "); \ print("coveragePercent =", branch_coverage["coveragePercent"]); \ print("linesCovered =", branch_coverage["linesCovered"]); \ print("linesMissed =", branch_coverage["linesMissed"]); \ print("linesTotal =", branch_coverage["linesTotal"]); \ print(""); \ \ is_ok = branch_coverage["linesMissed"] <= trunk_coverage["linesMissed"]; \ exit(0) if is_ok else exit(1) \ ' ```