add inf cv notebooks #59
Conversation
bogdan-galileo
commented
Jul 27, 2023
•
bogdan-galileo
commented
- Add inference cells on CV notebooks, EXCEPT the one used for the example runs (HF)
- Rename CV notebooks to
- HF_data for HuggingFace
- DF_data for DataFrame
- IF_data for ImageFolder
- Add all CV notebooks to CI + add minimize_for_CI
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
There was a problem hiding this comment.
@bogdan-galileo instead of overwriting the image classification examples, could we have a separate dir for inference? i.e. examples/image_classification_inference or examples/image_classification/inference?
.github/workflows/examples.yaml
Outdated
| @@ -152,7 +152,7 @@ jobs: | |||
| GALILEO_MODAL_TOKEN_SECRET: ${{ secrets.GALILEO_MODAL_TOKEN_SECRET }} | |||
| run: | | |||
| modal token set --token-id ${{ env.GALILEO_MODAL_TOKEN_ID }} --token-secret ${{ env.GALILEO_MODAL_TOKEN_SECRET }} | |||
| modal run ./scripts/populator_cuda.py::run --notebook-path examples/image_classification/Image_Classification_using_PyTorch_and_🔭_Galileo.ipynb | |||
| modal run ./scripts/populator_cuda.py::run --notebook-path examples/image_classification/Image_Classification_using_PyTorch_and_🔭_Galileo_IF_data.ipynb | |||
There was a problem hiding this comment.
this should not be removed because its for example runs.
There was a problem hiding this comment.
@anthonycorletti I didn't add an inference notebook by itself, I just added inference cells at the end of the normal notebooks. Also, renamed our notebooks since the names were a bit weird, it's now
- Galileo_HF_data for HuggingFace
- Galileo_IF_data for ImageFolder
- Galileo_DF_data for DataFrame
There was a problem hiding this comment.
I see, these will now fail in cloud since they have inference. Okay will fix that
.github/workflows/examples.yaml
Outdated
| @@ -258,7 +258,7 @@ jobs: | |||
| SLACK_WEBHOOK: ${{ secrets.ML_TEAM_EXAMPLES_WEBHOOK }} | |||
| SLACK_USERNAME: "Example Runs Root Content Populator" | |||
| SLACK_ICON_EMOJI: ":telescope:" | |||
| SLACK_COLOR: ${{ ((needs.mltc-example.result == 'success') && (needs.ner-example.result == 'success') && (needs.tc-example.result == 'success') && (needs.ic-example.result == 'success') && (needs.od-example.result == 'success') && (needs.semseg-example.result == 'success') && 'success') || 'failure' }} | |||
| SLACK_COLOR: ${{ ((needs.mltc-example.result == 'success') && (needs.ner-example.result == 'success') && (needs.tc-example.result == 'success') && (needs.ic-example-HF.result == 'success') && (needs.od-example.result == 'success') && (needs.semseg-example.result == 'success') && 'success') || 'failure' }} | |||
There was a problem hiding this comment.
same here, should be changed back. if there's an inference example run we can add a separate job in the dag.
| runs-on: ubuntu-latest | ||
| needs: paths-filter | ||
| if: ${{ github.event_name != 'pull_request' || (github.event_name == 'pull_request' && needs.paths-filter.outputs.ic-pytorch == 'true' ) }} |
There was a problem hiding this comment.
should not be deleted. copy paste errors?
