Skip to content

Conversation

@Red-Portal
Copy link
Member

@Red-Portal Red-Portal commented Aug 19, 2025

This PR separates the AD backend integration tests into their own workflow. This has the following benefit:

  • The test codes become simpler since we don't have to loop over the various AD backends
  • The test results can be viewed at a glance from the GitHub Actions page
  • We can monitor the AD backend support via GitHub badges.

All AD integration tests invoke .github/workflows/AutoDiffIntegration.yml, which implements GitHub's reusable workflows feature.

@github-actions
Copy link
Contributor

AdvancedVI.jl documentation for PR #190 is available at:
https://TuringLang.github.io/AdvancedVI.jl/previews/PR190/

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Results

Benchmark suite Current: 5928201 Previous: e801817 Ratio
normal/RepGradELBO + STL/meanfield/Zygote 3869106797 ns 4047146731.5 ns 0.96
normal/RepGradELBO + STL/meanfield/ReverseDiff 1128292103 ns 1118748915 ns 1.01
normal/RepGradELBO + STL/meanfield/Mooncake 1190705399 ns 1241121652 ns 0.96
normal/RepGradELBO + STL/fullrank/Zygote 3955271108 ns 4002369352.5 ns 0.99
normal/RepGradELBO + STL/fullrank/ReverseDiff 1676757225 ns 1619611799 ns 1.04
normal/RepGradELBO + STL/fullrank/Mooncake 1281361888 ns 1285127945.5 ns 1.00
normal/RepGradELBO/meanfield/Zygote 2767657958 ns 2834638479.5 ns 0.98
normal/RepGradELBO/meanfield/ReverseDiff 788761749 ns 788049327 ns 1.00
normal/RepGradELBO/meanfield/Mooncake 1062465846 ns 1093464215 ns 0.97
normal/RepGradELBO/fullrank/Zygote 2785921421.5 ns 2873772655.5 ns 0.97
normal/RepGradELBO/fullrank/ReverseDiff 972573810 ns 956640380.5 ns 1.02
normal/RepGradELBO/fullrank/Mooncake 1118695995 ns 1161037456 ns 0.96
normal + bijector/RepGradELBO + STL/meanfield/Zygote 5484603213 ns 5705749184 ns 0.96
normal + bijector/RepGradELBO + STL/meanfield/ReverseDiff 2380715424 ns 2472318329 ns 0.96
normal + bijector/RepGradELBO + STL/meanfield/Mooncake 4098330520.5 ns 4249849133.5 ns 0.96
normal + bijector/RepGradELBO + STL/fullrank/Zygote 5615396878 ns 5732121704 ns 0.98
normal + bijector/RepGradELBO + STL/fullrank/ReverseDiff 3047981874.5 ns 3070250822 ns 0.99
normal + bijector/RepGradELBO + STL/fullrank/Mooncake 4210618967 ns 4377541897 ns 0.96
normal + bijector/RepGradELBO/meanfield/Zygote 4309224236 ns 4461833085.5 ns 0.97
normal + bijector/RepGradELBO/meanfield/ReverseDiff 2030629168 ns 2088736319 ns 0.97
normal + bijector/RepGradELBO/meanfield/Mooncake 3930307061 ns 4122678327.5 ns 0.95
normal + bijector/RepGradELBO/fullrank/Zygote 4353748197 ns 4571360982 ns 0.95
normal + bijector/RepGradELBO/fullrank/ReverseDiff 2295184083 ns 2345239376 ns 0.98
normal + bijector/RepGradELBO/fullrank/Mooncake 4071153970.5 ns 4217237241 ns 0.97

This comment was automatically generated by workflow using github-action-benchmark.

@codecov
Copy link

codecov bot commented Aug 20, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@981bcc7). Learn more about missing BASE report.
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #190   +/-   ##
=======================================
  Coverage        ?   88.98%           
=======================================
  Files           ?       18           
  Lines           ?      445           
  Branches        ?        0           
=======================================
  Hits            ?      396           
  Misses          ?       49           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Red-Portal Red-Portal requested a review from penelopeysm August 21, 2025 16:25
test/runtests.jl Outdated
Comment on lines 57 to 58
if GROUP == "All" || GROUP == "General" || GROUP == "AD"
if GROUP == "AD"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this setup feels a bit complex. Could you just have two different tests: those which use AD and those which don't? In other words:

  • turn 'General' into 'AD' and maybe just rename it ReverseDiff.yml
  • group the families and paramspacesgd tests together

Copy link
Member Author

@Red-Portal Red-Portal Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree. I actually made things simpler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should go back to ReTest since it seems to be a little better maintained than before. The test filtering feature there is really useful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite agnostic towards frameworks like ReTest. If you think it's useful I wouldn't be opposed to it. I've been trying to use https://github.com/theogf/TestPicker.jl recently which (I think) does similar things too with test filtering

Red-Portal and others added 3 commits August 22, 2025 12:47
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@Red-Portal Red-Portal requested a review from penelopeysm August 22, 2025 17:32
Copy link
Member

@penelopeysm penelopeysm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good :)

@Red-Portal Red-Portal merged commit ead1554 into main Aug 22, 2025
28 of 34 checks passed
@Red-Portal Red-Portal deleted the AD_integration_tests branch August 22, 2025 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants