Skip to content

Modernize generator tests - #12216

Merged
sjrl merged 28 commits into
deepset-ai:mainfrom
DhanushPillay:modernize-generator-tests
Aug 6, 2026
Merged

Modernize generator tests#12216
sjrl merged 28 commits into
deepset-ai:mainfrom
DhanushPillay:modernize-generator-tests

Conversation

@DhanushPillay

Copy link
Copy Markdown
Contributor

Proposed Changes:

This PR modernizes the test/components/generators/ test suite by fully adopting strict mypy type hints.

I've gone through the generator tests and cleaned them up to make sure they're completely type-safe. Specifically, this includes:

  • Adding explicit return types (like -> None) and parameter types (e.g., using Any for fixtures and monkeypatches) across all the root generator and chat generator tests.
  • Refactoring assertions that previously relied on str | None so they are now strict-type compliant (e.g. safely checking .text).
  • Doing a full sweep to remove any redundant or unused # type: ignore comments that were cluttering up the codebase.
  • Finally, I've added the test/components/generators/ directory to the types check in pyproject.toml so these tests will be officially checked in the CI pipeline going forward.

How did you test it?

I verified everything locally by running hatch run test:types. The entire test/components/generators/ directory now passes with zero errors.

Notes for the reviewer

Since this is purely an internal update to the test suite and build config (no user-facing API or feature changes), I skipped generating a release note as per the contribution guidelines for CI/test changes.

@DhanushPillay
DhanushPillay requested a review from a team as a code owner August 1, 2026 10:24
@DhanushPillay
DhanushPillay requested review from sjrl and removed request for a team August 1, 2026 10:24
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

@DhanushPillay is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Aug 1, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Comment thread test/components/generators/chat/test_azure.py Outdated
Comment thread test/components/generators/chat/test_azure.py Outdated
Comment thread test/components/generators/chat/test_azure.py Outdated
Comment thread test/components/generators/chat/test_azure.py Outdated
@sjrl sjrl self-assigned this Aug 3, 2026
@DhanushPillay

Copy link
Copy Markdown
Contributor Author

Hey @sjrl , thanks for the review! I just pushed a commit with those updates.

I've added the specific error codes to all the type: ignore comments (like [attr-defined]) so they're no longer generic. For the message.text checks, I dropped the ignores and just added assert message.text is not None before using them, as you suggested. I also fixed a quick merge conflict in pyproject.toml and ran the formatter so the CI should be green now.

Let me know if everything looks good!

@DhanushPillay
DhanushPillay force-pushed the modernize-generator-tests branch from e0c9792 to 0c82941 Compare August 3, 2026 12:46
@DhanushPillay
DhanushPillay requested a review from sjrl August 3, 2026 12:48
Comment thread test/components/generators/chat/test_azure.py Outdated
@DhanushPillay
DhanushPillay requested a review from sjrl August 5, 2026 13:10
Comment thread test/components/generators/chat/test_openai_responses.py
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Coverage report

This PR does not seem to contain any modification to coverable code.

@DhanushPillay
DhanushPillay requested a review from sjrl August 5, 2026 15:05
Comment thread test/components/generators/chat/test_openai_responses.py Outdated
Comment thread test/components/generators/chat/test_fallback.py Outdated
Comment thread test/components/generators/chat/test_llm.py
@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @DhanushPillay, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Aug 6, 2026
@HaystackBot
HaystackBot marked this pull request as draft August 6, 2026 07:15
@DhanushPillay
DhanushPillay marked this pull request as ready for review August 6, 2026 07:17
@sjrl

sjrl commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@DhanushPillay sorry false positive with the bot.

@sjrl sjrl removed the cla-pending PR is in draft until the contributor signs the CLA label Aug 6, 2026
@DhanushPillay

Copy link
Copy Markdown
Contributor Author

@sjrl I have just pushed a commit to address the remaining review comments. I added the missing return types, replaced the type ignores in test_fallback.py with strict type hints, and kept the dynamic __haystack_*__ attributes as expected. I also ran ruff format to ensure all formatting is correct. Please let me know if anything else is needed

]
message = ChatMessage.from_tool(
tool_result=tool_result,
tool_result=tool_result, # type: ignore[arg-type]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why this one needs an ignore? The type for tool_result is ToolCallResultContentT = str | Sequence[TextContent | ImageContent | FileContent] which I thought matches the input correctly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right, it doesn't need the ignore! The issue was that without an explicit type annotation on the tool_result list, mypy inferred a narrower type (list[TextContent] or similar) instead of recognizing it as a valid Sequence[TextContent | ImageContent | FileContent]. I've fixed it by adding an explicit type hint on the variable, so mypy can see it matches ToolCallResultContentT directly. Thanks for catching that

@DhanushPillay

Copy link
Copy Markdown
Contributor Author

@sjrl Hey, I've addressed all the review feedback and CI looks good on my end, the only failing check is the Vercel deploy which seems unrelated. Happy to make any other changes if needed

@sjrl sjrl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@sjrl
sjrl merged commit f4ba8d1 into deepset-ai:main Aug 6, 2026
21 of 22 checks passed
@DhanushPillay
DhanushPillay deleted the modernize-generator-tests branch August 6, 2026 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants