feat: add log exporting to e2e tests#308
Merged
mergify[bot] merged 1 commit intoinstructlab:mainfrom Nov 13, 2024
Merged
Conversation
mergify
Bot
added
CI/CD
RobotSail
force-pushed
the
official-loss-printing
branch
from
4742edf to
8f77076
Compare
RobotSail
force-pushed
the
official-loss-printing
branch
from
8f77076 to
00e0231
Compare
RobotSail
force-pushed
the
official-loss-printing
branch
from
00e0231 to
4d3e3a7
Compare
RobotSail
requested review from
JamesKunstle,
Maxusmusti,
aldopareja,
cdoern,
danmcp and
nathan-weinberg
danmcp
suggested changes
Oct 25, 2024
RobotSail
force-pushed
the
official-loss-printing
branch
from
4d3e3a7 to
82d5711
Compare
danmcp
reviewed
Oct 26, 2024
nathan-weinberg
requested changes
Oct 29, 2024
RobotSail
force-pushed
the
official-loss-printing
branch
2 times, most recently
from
1fd7c48 to
387828b
Compare
RobotSail
force-pushed
the
official-loss-printing
branch
from
387828b to
039b743
Compare
Member
Author
|
@nathan-weinberg I've updated the CI scripts with your feedback, please take another pass when you get a chance and make sure that we didn't miss anything. |
nathan-weinberg
approved these changes
Nov 13, 2024
Member
nathan-weinberg
left a comment
nathan-weinberg
left a comment
There was a problem hiding this comment.
I'd like the version number commenting to be consistent with how is it everywhere else, but otherwise LGTM
danmcp
approved these changes
Nov 13, 2024
Member
|
Can we squash commits before merging? Great work on this @RobotSail excited to see it in action! |
Currently, the training library runs through a series of end-to-end tests which ensure there are no bugs in the code being tested. However; we do not perform any form of validation to assure that the training logic and quality has not diminished. This presents an issue where we can potentially be "correct" in the sense of no hard errors being hit, but invisible bugs may be introduced which cause models to regress in training quality, or other bugs that plague the models themselves to seep in. This commit fixes that problem by introducng the ability to export the training loss data itself from the test and rendering the loss curve using matplotlib. Signed-off-by: Oleg S <97077423+RobotSail@users.noreply.github.com>
RobotSail
force-pushed
the
official-loss-printing
branch
from
ab6151d to
c809c73
Compare
Member
Author
|
@nathan-weinberg This has been squashed, I'll remove the hold since that's the only issue. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, the training library runs through a series of end-to-end tests which ensure there are
no bugs in the code being tested. However; we do not perform any form of validation to assure that
the training logic and quality has not diminished.
This presents an issue where we can potentially be "correct" in the sense of no hard errors being hit,
but invisible bugs may be introduced which cause models to regress in training quality, or other
bugs that plague the models themselves to seep in.
This commit fixes that problem by introducng the ability to export the training loss data itself
from the test and rendering the loss curve using matplotlib.
When the results are outputted, they can be found under the "Summary" tab of a Github actions run.
For example:
Resolves #179
Signed-off-by: Oleg S 97077423+RobotSail@users.noreply.github.com