Skip to content

Added a step to upload to Amazon S3.#326

Merged
xuzhao9 merged 1 commit into
pytorch:masterfrom
xuzhao9:xz9/add-s3-upload
Mar 23, 2021
Merged

Added a step to upload to Amazon S3.#326
xuzhao9 merged 1 commit into
pytorch:masterfrom
xuzhao9:xz9/add-s3-upload

Conversation

@xuzhao9

@xuzhao9 xuzhao9 commented Mar 22, 2021

Copy link
Copy Markdown
Contributor

No description provided.

path: benchmark-result-*.json
- name: Upload results to Amazon S3
run: |
LAST_MODIFIED_DIR=$(find ${HOME}/benchmark-results-v0.1/ -mindepth 1 -maxdepth 1 -type d -exec stat -t {} \; | sort -r -n -k 13,13 | head -1 | cut -d " " -f 1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this LAST_MODIFED_DIR command very safe? would the uploader do something bad if LAST_MODIFIED_DIR is a duplicate of one uploaded before? or if it is empty?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If LAST_MODIFIED_DIR is a duplicate, the uploader will upload the file again. Since --torchbench-result-dir is a required argument, if it is empty the upload_s3.py will fail with error.

@xuzhao9
xuzhao9 merged commit 828bd21 into pytorch:master Mar 23, 2021
@xuzhao9
xuzhao9 deleted the xz9/add-s3-upload branch March 23, 2021 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants