Skip to content

ODSC-46634/utilize oci UploadManager to upload model artifacts #304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 5, 2023

Conversation

mingkang111
Copy link
Member

@mingkang111 mingkang111 commented Aug 28, 2023

Description

Utilize the UploadManager in the ArtifactUploader to upload model artifacts to the object storage.
https://jira.oci.oraclecorp.com/browse/ODSC-46634

For details see this doc.

User experience

  • Adds argument PARALLEL_PROCESS_COUNT in the following public APIs.
ads.model.GenericModel.save
ads.model.GenericModel.upload_artifact
  • After this change, the time used in save() for large model artifacts was reduced by ~40%.
    **Test with model artifact ~=2.3G
Screenshot 2023-09-05 at 9 58 43 AM

Screenshot 2023-09-05 at 12 32 52 PM

Pre-commit

Screenshot 2023-09-05 at 9 57 19 AM

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Aug 28, 2023
@github-actions
Copy link

📌 Cov diff with main:

Coverage-19%

📌 Overall coverage:

Coverage-32.34%

@github-actions
Copy link

📌 Cov diff with main:

Coverage-87%

📌 Overall coverage:

Coverage-71.67%

@mingkang111 mingkang111 added the wip Work in progress label Aug 29, 2023
@mingkang111 mingkang111 removed the wip Work in progress label Sep 5, 2023
mrDzurb
mrDzurb previously approved these changes Sep 5, 2023
@mingkang111 mingkang111 marked this pull request as ready for review September 5, 2023 19:19
@github-actions
Copy link

github-actions bot commented Sep 5, 2023

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@github-actions
Copy link

github-actions bot commented Sep 5, 2023

📌 Cov diff with main:

Coverage-64%

📌 Overall coverage:

Coverage-68.46%

@github-actions
Copy link

github-actions bot commented Sep 5, 2023

📌 Cov diff with main:

Coverage-81%

📌 Overall coverage:

Coverage-71.68%

@mingkang111 mingkang111 requested review from z7ye and lu-ohai September 5, 2023 21:20
@mingkang111 mingkang111 merged commit d9756a8 into main Sep 5, 2023
@mingkang111 mingkang111 deleted the ODSC-46634/Utilize_oci_to_upload_model_artifacts branch September 5, 2023 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants