Quickly plot sweeps #12
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jun 27, 2023
fxmarty
reviewed
Jun 28, 2023
| dynamic = args.compile == "dynamic" | ||
| fullgraph = args.compile == "fullgraph" | ||
| model.forward = torch.compile(model.forward, mode="reduce-overhead", fullgraph=fullgraph, dynamic=dynamic) | ||
| mode = "reduce-overhead" if not args.sweep else "max-autotune" |
Collaborator
Author
There was a problem hiding this comment.
To see how fast we can get with compile :) Happy to revert if you feel like it won't be the planned use case!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@fxmarty this PR adds tools to quickly run and plot sweeps, so we can make deeper comparisons between commits in an automated way.
In particular, we can now trigger a sweep with
--sweep batchor--sweep length, to which triggers a sweep over batch size or prompt length, respectively, storing the result in a dataframe. The results can quickly be plotted with e.g.python scripts/plot_results.py --sweep length, which stores a plot to./results/NOTE: I've triggered
max-autotuneduring a sweep, so we can see how fast it can go in its compiled form 🚤Results
Here are the plots for the current
main, on a RTX 3090Batch size sweep
Prompt length sweep