Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add back hardcoded default templates #4998

Merged
merged 1 commit into from
May 24, 2023
Merged

feat: Add back hardcoded default templates #4998

merged 1 commit into from
May 24, 2023

Conversation

silvanocerza
Copy link
Contributor

@silvanocerza silvanocerza commented May 23, 2023

Proposed Changes:

Add the hardcoded PromptTemplate remove with PR #4879.

How did you test it?

Add a new test that verifies the expected legacy template is returned when using legacy template names.

Notes for the reviewer

There's more information in the comment at the top of legacy_default_templates.py.

Copy link
Member

@TuanaCelik TuanaCelik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JFYI, the PR merged by @ZanSara renames prompt_text to prompt and removes name in PromptTemplate.

@silvanocerza
Copy link
Contributor Author

@TuanaCelik yep, I noticed. I'm fixing it right now.

@silvanocerza silvanocerza changed the title Add back hardcoded default templates feat: Add back hardcoded default templates May 23, 2023
@silvanocerza silvanocerza marked this pull request as ready for review May 23, 2023 15:35
@silvanocerza silvanocerza requested a review from a team as a code owner May 23, 2023 15:35
@silvanocerza silvanocerza requested review from julian-risch, masci and ZanSara and removed request for a team May 23, 2023 15:35
@coveralls
Copy link
Collaborator

coveralls commented May 23, 2023

Pull Request Test Coverage Report for Build 5059281109

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 44 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.04%) to 39.549%

Files with Coverage Reduction New Missed Lines %
nodes/prompt/prompt_node.py 44 59.34%
Totals Coverage Status
Change from base Build 5059192256: 0.04%
Covered Lines: 8855
Relevant Lines: 22390

💛 - Coveralls

@ZanSara
Copy link
Contributor

ZanSara commented May 24, 2023

Ok, I see the value of these, but my concerns with this approach is: do we want to keep these local PromptTemplates forever, or not?

I see there's no deprecation schedule or hint they will eventually be deprecated. I'm ok with keeping them, but if so, let's not call them "legacy" or similar names: we should find a stable place for them that makes sense. Also, I'd rather distinguish them strongly from the ones in the hub: question-answering and deepset/question-answering are too similar to trigger such different behavior in PromptTemplate. Right now, basically one works as expected (the first) and the other does not.

If we don't want to support output parsers in the hub, then we either go head-in with this decision and deprecate these prompts, or backtrack and rethink the integration. And to be honest I believe output parsers could be supported in the hub (I'm volunteering for this task if necessary) so, if we need them so badly, we should consider this option too.

In short, I see this solution here as valid only if it's a patch to ease the transition. Let's clarify what's our stand on this.

Copy link
Contributor

@ZanSara ZanSara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's clarify what we want the API to look like before moving forward

@ZanSara
Copy link
Contributor

ZanSara commented May 24, 2023

Ok after some discussion, let's add a deprecation notice to these templates. It should be enough for now.

Copy link
Contributor

@ZanSara ZanSara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be enough to clarify the situation imho.

Copy link
Contributor

@ZanSara ZanSara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this is probably good like this. We'll consider if deprecating the default templates or not later, depending on usage too. No need to block the hotfix for that 🙂

@ZanSara ZanSara merged commit 56d033e into main May 24, 2023
86 checks passed
@ZanSara ZanSara deleted the default-templates branch May 24, 2023 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants