Skip to content

Replace nextest experimental JSON output with JUnit XML for CI test reporting#132

Merged
jamals86 merged 3 commits intomainfrom
copilot/update-nextest-output-format
Feb 11, 2026
Merged

Replace nextest experimental JSON output with JUnit XML for CI test reporting#132
jamals86 merged 3 commits intomainfrom
copilot/update-nextest-output-format

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 11, 2026

The CI test job fails because NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 produces compilation output instead of test results, causing badge generation to report 0 tests.

Changes

Created .config/nextest.toml

Defines [profile.ci] with JUnit XML output:

[profile.ci]
junit.path = "junit.xml"
failure-output = "immediate-final"
fail-fast = false
retries = 1

Updated .github/workflows/ci.yml

  • Removed: Experimental --message-format libtest-json-plus and grep/awk JSON parsing
  • Added: --profile ci flag to generate JUnit XML at target/nextest/ci/junit.xml
  • Added: Python XML parser using xml.etree.ElementTree to extract test counts
  • Added: dorny/test-reporter@v1 for GitHub test annotations
  • Split: Test execution and badge generation into separate steps with proper error handling

The new workflow generates reliable test badges and GitHub annotations from standard JUnit XML format instead of relying on experimental JSON output.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • get.nexte.st
    • Triggering command: /usr/bin/curl curl -LsSf REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

Problem

The CI workflow test job is failing because cargo nextest is not producing valid libtest-json-plus output. The current workflow tries to parse JSON test results, but the results.json file only contains compilation output, not test results.

Failing job: https://github.com/jamals86/KalamDB/actions/runs/21884719493/job/63176530872

Error:

Tests: 0 passed, 0 failed (total: 0)
Error: No tests found. Parsing likely failed.

Solution

Replace the current JSON-based approach with JUnit XML reports, which is the standard format for nextest. This provides:

  • Reliable test result parsing
  • Better GitHub integration with test-reporter
  • Cleaner code without complex grep/awk pipelines

Required Changes

1. Create nextest configuration file

Create .config/nextest.toml:

[profile.ci]
# Store test results in JUnit format
junit.path = "junit.xml"

# Show output for failed tests immediately
failure-output = "immediate-final"

# Don't fail fast - run all tests
fail-fast = false

# Retry flaky tests once
retries = 1

2. Update .github/workflows/ci.yml

Replace the test job (lines 131-241) with:

  test:
    name: Test Suite
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Install Rust toolchain
        uses: dtolnay/rust-toolchain@stable
        with:
          toolchain: ${{ env.RUST_VERSION }}

      - name: Install system dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y --no-install-recommends \
            clang \
            libclang-dev \
            pkg-config \
            libssl-dev

      - name: Cache cargo registry
        uses: actions/cache@v4
        with:
          path: ~/.cargo/registry
          key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
          restore-keys: |
            ${{ runner.os }}-cargo-registry-

      - name: Cache cargo index
        uses: actions/cache@v4
        with:
          path: ~/.cargo/git
          key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.lock') }}
          restore-keys: |
            ${{ runner.os }}-cargo-index-

      - name: Cache cargo build
        uses: actions/cache@v4
        with:
          path: target
          key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }}
          restore-keys: |
            ${{ runner.os }}-cargo-build-target-

      - name: Install cargo-nextest
        run: |
          curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin

      - name: Run tests (nextest)
        run: |
          cargo nextest run --profile ci --workspace --no-fail-fast --failure-output immediate-final
        continue-on-error: true

      - name: Parse test results and generate badge
        if: always()
        run: |
          JUNIT_FILE="target/nextest/ci/junit.xml"
          
          if [ ! -f "$JUNIT_FILE" ]; then
            echo "Error: JUnit XML report not found at $JUNIT_FILE"
            exit 1
          fi
          
          # Parse JUnit XML using Python (available in ubuntu-latest)
          eval $(python3 << 'PYTHON_SCRIPT'
          import xml.etree.ElementTree as ET
          import sys
          
          try:
              tree = ET.parse("target/nextest/ci/junit.xml")
              root = tree.getroot()
              
              total = 0
              failures = 0
              errors = 0
              
              # Sum up all testsuite elements
              for testsuite in root.findall('.//testsuite'):
                  total += int(testsuite.get('tests', 0))
                  failures += int(testsuite.get('failures', 0))
                  errors += int(testsuite.get('errors', 0))
              
              failed = failures + errors
              passed = total - failed
              
              print(f"TOTAL={total}")
              print(f"PASSED={passed}")
              print(f"FAILED={failed}")
              
              if total == 0:
                  print("Error: No tests found in JUnit report", file=sys.stderr)
                  sys.exit(1)
                  
          except Exception as e:
              print(f"Error parsing JUnit XML: {e}", file=sys.stderr)
              sys.exit(1)
          PYTHON_SCRIPT
          )
          
          echo "Tests: $PASSED passed, $FAILED failed (total: $TOTAL)"
          
          # Determine badge color and message
          if [ "$FAILED" -eq "0" ]; then
            COLOR="brightgreen"
            MESSAGE="$PASSED/$TOTAL passed"
          else
            COLOR="red"
            MESSAGE="$PASSED/$TOTAL passed, $FAILED failed"
          fi
          
          echo "Badge: $MESSAGE ($COLOR)"
          
          # Generate badge JSON
          mkdir -p .github/badges
          cat > .github/badges/tests.json << EOF
          {
            "schemaVersion": 1,
            "label": "tests",
            "me...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/jamals86/KalamDB/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 11, 2026 07:07
- Create .config/nextest.toml with CI profile for JUnit output
- Update CI workflow to use JUnit XML instead of experimental JSON
- Add Python-based JUnit XML parsing for test badge generation
- Add dorny/test-reporter for GitHub test annotations
- Split test execution and badge generation into separate steps

Co-authored-by: jamals86 <137282926+jamals86@users.noreply.github.com>
Co-authored-by: jamals86 <137282926+jamals86@users.noreply.github.com>
Copilot AI changed the title [WIP] Replace cargo nextest output with JUnit XML reports Replace nextest experimental JSON output with JUnit XML for CI test reporting Feb 11, 2026
Copilot AI requested a review from jamals86 February 11, 2026 07:11
@jamals86 jamals86 marked this pull request as ready for review February 11, 2026 07:17
@jamals86 jamals86 merged commit 1ccd3b3 into main Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants