Part 2: Creating New Loss Function Classes #159
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sociated tests. These will be used in future PRs to move PerFCL, FENDA, and other clients to use these losses.
…oss_function_classes
emersodb
changed the title
…oss_function_classes
…oss_function_classes
sanaAyrml
reviewed
Jun 3, 2024
Collaborator
sanaAyrml
left a comment
sanaAyrml
left a comment
There was a problem hiding this comment.
Overall looks good, just proposed some minor changes.
… machine, Fixing a PICAI mypy error as well.
sanaAyrml
approved these changes
Jun 4, 2024
jewelltaylor
reviewed
Jun 4, 2024
jewelltaylor
reviewed
Jun 4, 2024
jewelltaylor
reviewed
Jun 4, 2024
jewelltaylor
reviewed
Jun 4, 2024
jewelltaylor
reviewed
Jun 4, 2024
…nting improvements, adding a small test and fixing a test.
jewelltaylor
reviewed
Jun 5, 2024
| negative_similarity = self.cos_sim(features, negative_pair) | ||
| logits = torch.cat((logits, negative_similarity.reshape(-1, 1)), dim=1) | ||
|
|
||
| # Compute the similarity of the batch of features with the collection of batches of negative pair features |
jewelltaylor
approved these changes
Jun 5, 2024
Contributor
jewelltaylor
left a comment
jewelltaylor
left a comment
There was a problem hiding this comment.
Thanks for the changes, looks great!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Type
Feature
Short Description
Clickup Ticket(s): Link
This PR is Part 1 of several PRs aiming to simplify and decouple our implementations of FENDA and PerFCL. In this first part, I'm simply creating loss function classes that will be used by the different clients. These functions are stand-alone in this PR to be easier to review.
There are also some small documentation changes and an variable name change for the Ditto and MR-MTL clients.
Tests Added
Stand-along tests for each of the loss functions implemented, along with one for the pre-existing drift loss function.