fix: pin uncertainty and context-attribution revisions and update uncertai…

[AngeloDanducci]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes bug: test_canned_input[uncertainty] fails due to unpinned/outdated HF repo revision #969

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[jakelorocco]

@frreiss, should we pin revisions for the intrinsics tests? That way updates to the adapters don't immediately impact our testing infrastructure? Or do we think that there's a risk that the adapters get updated but Mellea doesn't?

[markstur]

LGTM and the temperature add is needed to fix CI.

Re: pinning the version, we can do what Fred recommends but at least we already know our CI is breakable with a small config change.

[jakelorocco]

I was thinking more about the version pinning. My suggestion: I think we can xfail these tests when the revision changes. We can also add a new test that runs only during nightlies / outside of CICD that fails if any of the revisions changes. That way our CICD pipeline is less brittle but we are still aware of when changes happen.

[AngeloDanducci]

I like both of those ideas better - I'll swap over to the xfail shortly.

[frreiss]

@frreiss, should we pin revisions for the intrinsics tests? That way updates to the adapters don't immediately impact our testing infrastructure? Or do we think that there's a risk that the adapters get updated but Mellea doesn't?

If we pin every intrinsic to a different long string of base64 commit hash, we will quickly forget what we were doing. Maintenance will require getting every model’s owner on the line on a regular basis to determine what is the right commit hash to pin each intrinsic to this week. People will not want to go through that hassle and the files on the head of the main branch will become incompatible with Mellea. People who use the intrinsics through high level APIs will get outdated models in their applications and won’t understand why.

Maybe tag the release and pin the tag instead?