Skip to content

LOO refactoring and adding pred_measure feature#363

Draft
florence-bockting wants to merge 2 commits into
masterfrom
pred_measure
Draft

LOO refactoring and adding pred_measure feature#363
florence-bockting wants to merge 2 commits into
masterfrom
pred_measure

Conversation

@florence-bockting
Copy link
Copy Markdown
Contributor

@florence-bockting florence-bockting commented Jun 2, 2026

Fixes #281

This PR is work in progress and the description will be refined over time.

Description

This PR is a larger project with the aim to refactor the loo package and to add new features related to a new pred_measure family.

In the course of refactoring this PR aims to address also the following PRs:

How to navigate this PR

As this is a large, long-running PR it might help to first read through the supporting documents that explain the design, scope, and current status:

  • developer-notes.md: The central reference for this project: current status, open tasks, outstanding design decisions, and known blockers. Start here.
  • pred_measure-formulas.Rmd: An overview of all supported scores and metrics, including their mathematical definitions.
  • pred_measure-family.Rmd: A conceptual introduction to the new pred_measure family and its intended usage.

Acknowledgements

This work builds on an initial prototype of the new pred_measure feature implemented by @VisruthSK under guidance from @jgabry (see VisruthSK/loo-sandbox).

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 2, 2026

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 55b80cd is merged into master:

  • ✔️loo_function: 1.75s -> 1.75s [-0.27%, +0.16%]
  • 🚀loo_matrix: 1.29s -> 1.28s [-1.45%, -0.62%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment