Sequence to Sequence Problem Type #308
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maxjeblick
reviewed
Jul 27, 2023
There was a problem hiding this comment.
Thanks a lot for adding this problem type, it looks in a very good state!
I would move prepare_lora and generate to separate functions to avoid code duplication (also, they will probably be needed for other potential problem types as well).
Apart from that, the code looks fine, will test it more the next days.
|
Thx for the comments @maxjeblick - tried to address them. |
maxjeblick
reviewed
Jul 31, 2023
maxjeblick
suggested changes
Jul 31, 2023
There was a problem hiding this comment.
Thanks for the changes! Some observations:
- Summary tab fails with
AttributeError: 'ConfigProblemBase' object has no attribute 'hf' - Train Data Insights currently shows decoded input ids, whereas the model uses prompt input ids. Maybe it is useful in general to show prompt input ids, instead of input ids? Could potentially also changed in Max/insights table view #301.
- Mid-term, promoting
generateandprepare_lorato dedicated functions (with corresponding arguments,cfg,backbone, ...) may be useful. We can leave it here as is and I can have a look in [CODE IMPROVEMENT] Promote RLHF as a separate problem type #317.
Merged
|
The PR is probably good to merge after hf has been added to the config. |
|
Yes, I just noticed that I also need to adjust the summary model card now. On it. |
maxjeblick
approved these changes
Aug 3, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #100
Adds a new problem Type Sequence to Sequence Modeling. For now tested for different T5 models.
We can probably go step by step adding more functionality if this will be frequently used.
I tried to reuse as much as possible to not have much duplicate code.
@maxjeblick please take an initial look, it might need 1-2 iterations
Todo: