Bert qa #50
Bert qa #50
Conversation
There was a problem hiding this comment.
Great job on getting this integration working and in script form. Really excited about the results, and I had a good time using your previous notebook.
Took a first pass and left some comments. Didn't comment too much on style because running make style will fix a lot of the formatting. Overall, it would be great if we could structure the readme and script to demonstrate as simple as possible how to integrate sparseML into transformers, pointing this out in both documents.
|
Done with code changes. Working on Readme changes @bfineran can you give the code another look through for comments/thoughts? README.md will have to wait until all the experiments finish. |
There was a problem hiding this comment.
A few things at the high level and left comments:
- let's move this over to a new folder from examples to integrations (the timm and ultrallytics are being migrated there as well)
- let's rename prune_config_files to recipes
-- bonus if we can get these models and recipes pushed to thee sparsezoo instead of storing here - let's change the formats of the recipes to markdown from yaml and include some info in them such as the example command to run (for when they are pushed to sparsezoo)
|
Changes implemented. Upgraded to new transformers library and getting a blocking error. Opened an issue on their page. huggingface/transformers#10618 |
|
Fixed issue with transformers. Ready for merge. |
There was a problem hiding this comment.
As discussed this is in a good state to land for a research integration. Over the next weeks, we need to nail down exact flows and recipes for users (as well as improving the recipes) and then push those to the SparseZoo/potentially huggingface and remove the recipes and table from this integration.
Additionally, as we make things easier we'll need to evaluate how close we can make run_qa.py to the huggingface integration. Layer dropping, distillation, etc should all be added in a separate file.
* Correct deepsparse highlight images * update highlights, tutorials, and infographic sizes
4 participants
Adding example for BERT SQUAD with onnx export.