Skip to content

Conversation

@jakelorocco
Copy link
Contributor

@jakelorocco jakelorocco commented Aug 8, 2025

A few tests were consistently failing even though the code was executing correctly. I've updated the tests to fix that.

One notable change here is the addition of a Granite 3.3 Alora. While testing the failing hugging face test for the granite constraint alora, it was suggested that part of the issue might be that the 3.2 alora is generally worse than the 3.3 alora, especially because the output that our alora was running against contained unclosed quotes and punctuation that may have been impacting what the alora generated.

Ultimately, it looks like 3.3 still struggles with that as well (even with it's new syntax). But I had already implemented most of the changes, so I kept that.

To fix the failed test, I just ensured the test generated a smaller output that wasn't abruptly ended.

End result:

  • Watsonx tests pass but we still need to set up a dedicated account/runner
  • I could not replicate the ollama failures but preventatively marked as may fail
  • HuggingFace alora tests pass
  • ModelOptions tests pass

@jakelorocco jakelorocco marked this pull request as ready for review August 8, 2025 15:33
Copy link
Contributor

@nrfulton nrfulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good; thanks.

@jakelorocco jakelorocco merged commit a744798 into main Aug 8, 2025
@jakelorocco jakelorocco deleted the jal/fix-tests branch August 8, 2025 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants