Skip to content

Conversation

@jamesbhobbs
Copy link
Contributor

@jamesbhobbs jamesbhobbs commented Oct 10, 2025

feat: add codecov integration with test results upload

Summary

Adds codecov integration to the CI pipeline to upload coverage reports and test results. The implementation follows the same pattern as deepnote-internal and deepnote-toolkit repositories.

Changes:

  • .codecov.yml: Added codecov configuration with project and patch coverage checks (0.05% threshold, auto target)
  • jest.config.js: Added jest-junit reporter to output test results in JUnit XML format
  • .github/workflows/build.yml:
    • Updated pytest command to output XML coverage (--cov-report=xml) and test results (--junit-xml=coverage/pytest-results.xml)
    • Added codecov upload steps for both coverage and test results
    • Set fail_ci_if_error: false to prevent blocking CI if codecov has issues

Review & Testing Checklist for Human

  • Verify CODECOV_TOKEN secret is configured in repository settings at https://github.com/deepnote/jupyterlab-deepnote/settings/secrets/actions (this is required for codecov uploads to work)
  • Check that codecov.io project exists for this repository and is properly configured
  • Run CI and verify file paths are correct - confirm that the codecov upload steps find the expected files (coverage.xml, coverage/lcov.info, coverage/pytest-results.xml, coverage/junit.xml)
  • Verify test results XML generation - check that both pytest and jest are actually outputting test results in JUnit XML format as expected

Notes

This PR implements codecov integration as requested. The configuration matches the patterns used in other Deepnote repositories (deepnote-internal and deepnote-toolkit). The jest-junit package was already available in yarn.lock, so I configured it as a reporter.

If CODECOV_TOKEN is not configured, the codecov upload steps will fail but won't block CI due to fail_ci_if_error: false setting.

Link to Devin run: https://app.devin.ai/sessions/469dd0ba825c4715b7080a86ea978fcb
Requested by: @jamesbhobbs

Summary by CodeRabbit

  • Chores

    • Added explicit coverage thresholds and integrated automated coverage and test-result uploads into CI to enforce quality gates and fail on errors.
    • Centralized creation and submission of coverage and test artifacts for improved pipeline visibility.
  • Tests

    • Enabled JUnit-style test reporting for machine-readable results.
    • Standardized coverage report generation across test suites to support consistent quality checks.

- Add .codecov.yml configuration with project and patch coverage checks (0.05% threshold)
- Configure jest to output test results in JUnit XML format using jest-junit reporter
- Update pytest to output coverage in XML format and test results in JUnit XML format
- Add codecov/codecov-action@v5 step to upload coverage from both jest (lcov) and pytest (xml)
- Add codecov/test-results-action@v1 step to upload test results from both test suites
- Set fail_ci_if_error: false for coverage upload to prevent blocking CI if codecov has issues
- Use if: '!cancelled()' for test results upload to ensure results are uploaded even if tests fail

Follows the same codecov configuration pattern as deepnote-internal and deepnote-toolkit repos.

Note: CODECOV_TOKEN secret needs to be configured in repository settings for codecov uploads to work.
@devin-ai-integration
Copy link

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 10, 2025

📝 Walkthrough

Walkthrough

Adds a Codecov config (.codecov.yml) setting project and patch coverage status targets to auto with a 0.05% threshold. Updates CI (.github/workflows/build.yml) to collect coverage and test artifacts: run pytest with coverage and JUnit XML output, run Jest with coverageDirectory ./coverage and jest-junit reporter writing coverage/junit.xml, upload coverage via codecov-action@v5, and upload test results via codecov/test-results-action@v1 (gated by !cancelled()). Adds Jest config entries coverageDirectory: './coverage' and reporters including jest-junit.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    actor Dev as Developer
    participant GH as GitHub Actions
    participant Py as Pytest
    participant Je as Jest
    participant CC as Codecov

    Dev->>GH: push / open PR triggers build workflow
    GH->>Py: Run pytest --cov jupyterlab_deepnote\n--cov-report=xml --junit-xml=coverage/pytest-results.xml
    Py-->>GH: coverage.xml + coverage/pytest-results.xml

    GH->>Je: Run Jest (coverageDirectory='./coverage')\nreporters: default + jest-junit -> coverage/junit.xml
    Je-->>GH: coverage files + coverage/junit.xml

    GH->>CC: codecov-action@v5 upload (coverage.xml, coverage/lcov.info)
    alt workflow not cancelled
        GH->>CC: codecov/test-results-action@v1 upload (coverage/pytest-results.xml, coverage/junit.xml)
    end
    Note right of CC #D7EAF3: .codecov.yml sets\nproject & patch target: auto\nthreshold: 0.05%
Loading

Possibly related PRs

  • deepnote/deepnote-internal#18073: Same .codecov.yml change—sets target: auto and threshold: 0.05%.
  • deepnote/deepnote-internal#18477: Adds codecov/test-results-action and adjusts CI test-results upload steps.
  • ci: 80 percent codedov target deepnote#22: Alters .codecov.yml coverage.status entries for project and patch similarly.

Pre-merge checks

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title clearly and succinctly describes the main change by indicating the addition of Codecov integration along with test results upload, directly reflecting the pull request’s primary objective.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5a4afbd and 54f451d.

📒 Files selected for processing (1)
  • .github/workflows/build.yml (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: build
  • GitHub Check: check_release

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7ac4a8e and d02171e.

📒 Files selected for processing (3)
  • .codecov.yml (1 hunks)
  • .github/workflows/build.yml (2 hunks)
  • jest.config.js (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-09-08T23:11:17.867Z
Learnt from: CR
PR: deepnote/deepnote-toolkit#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-08T23:11:17.867Z
Learning: Use the provided code coverage commands and thresholds

Applied to files:

  • .codecov.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: check_release
  • GitHub Check: build
🔇 Additional comments (1)
.codecov.yml (1)

1-10: LGTM: Codecov thresholds align with team standards.

Config is valid. The 0.05% threshold is lenient but matches established practices.

Based on learnings.

- Create coverage directory before pytest runs to ensure junit xml output succeeds
- Fix codecov upload file paths (coverage.xml and coverage/lcov.info)
- Configure jest-junit to output to coverage/junit.xml explicitly
- Set coverageDirectory to ./coverage for consistency

Addresses CodeRabbit review comments.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 884fdb9 and 5a4afbd.

📒 Files selected for processing (1)
  • .github/workflows/build.yml (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: build
  • GitHub Check: check_release

@jamesbhobbs jamesbhobbs marked this pull request as ready for review October 10, 2025 10:51
@codecov
Copy link

codecov bot commented Oct 10, 2025

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@jamesbhobbs jamesbhobbs enabled auto-merge (squash) October 10, 2025 10:51
coderabbitai[bot]
coderabbitai bot previously approved these changes Oct 10, 2025
Add fail_ci_if_error: true to test results upload step to ensure CI fails when codecov upload fails, matching the coverage upload configuration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants