Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

oras pull will create a temporary file in the temp folder and will not delete it. #815

Closed
1 task done
chi-qin opened this issue Feb 15, 2023 · 3 comments · Fixed by #831
Closed
1 task done

oras pull will create a temporary file in the temp folder and will not delete it. #815

chi-qin opened this issue Feb 15, 2023 · 3 comments · Fixed by #831
Assignees
@chi-qin
Labels
bug Something isn't working good first issue Good for newcomers
Milestone
v1.0.0-rc.2

Comments

@chi-qin
Copy link
Contributor

chi-qin commented Feb 15, 2023

What happened in your environment?

Here is the log:

PS C:\Users\chiqin> oras pull -v -d docsbuildworkeracrpubdev.azurecr.io/loc-build:1.0.02230.181
time=2023-02-15T15:58:40+08:00 level=debug msg= Request URL: "https://docsbuildworkeracrpubdev.azurecr.io/v2/loc-build/manifests/1.0.02230.181"
time=2023-02-15T15:58:40+08:00 level=debug msg= Request method: "GET"
time=2023-02-15T15:58:40+08:00 level=debug msg= Request headers:
time=2023-02-15T15:58:40+08:00 level=debug msg=   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
time=2023-02-15T15:58:40+08:00 level=debug msg=   "User-Agent": "oras/0.16.0"
time=2023-02-15T15:58:42+08:00 level=debug msg= Response Status: "401 Unauthorized"
time=2023-02-15T15:58:42+08:00 level=debug msg= Response headers:
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Content-Type": "application/json; charset=utf-8"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Www-Authenticate": "Bearer realm=\"https://docsbuildworkeracrpubdev.azurecr.io/oauth2/token\",service=\"docsbuildworkeracrpubdev.azurecr.io\",scope=\"repository:loc-build:pull\""
time=2023-02-15T15:58:42+08:00 level=debug msg=   "X-Ms-Correlation-Request-Id": "36844dee-1046-4b78-96f9-a982321a198c"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Access-Control-Expose-Headers": "Docker-Content-Digest, WWW-Authenticate, Link, X-Ms-Correlation-Request-Id"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Docker-Distribution-Api-Version": "registry/2.0"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Server": "openresty"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Connection": "keep-alive"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "X-Content-Type-Options": "nosniff"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Date": "Wed, 15 Feb 2023 07:58:42 GMT"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Content-Length": "203"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Strict-Transport-Security": "max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains"
time=2023-02-15T15:58:42+08:00 level=debug msg= Request URL: "https://docsbuildworkeracrpubdev.azurecr.io/oauth2/token"
time=2023-02-15T15:58:42+08:00 level=debug msg= Request method: "POST"
time=2023-02-15T15:58:42+08:00 level=debug msg= Request headers:
time=2023-02-15T15:58:42+08:00 level=debug msg=   "Content-Type": "application/x-www-form-urlencoded"
time=2023-02-15T15:58:42+08:00 level=debug msg=   "User-Agent": "oras/0.16.0"
time=2023-02-15T15:58:43+08:00 level=debug msg= Response Status: "200 OK"
time=2023-02-15T15:58:43+08:00 level=debug msg= Response headers:
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Date": "Wed, 15 Feb 2023 07:58:44 GMT"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "X-Ms-Ratelimit-Remaining-Calls-Per-Second": "333.316667"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Strict-Transport-Security": "max-age=31536000; includeSubDomains"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Connection": "keep-alive"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "X-Ms-Correlation-Request-Id": "a63fb9c9-693a-4e82-ad6c-bbb058dd46cd"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Server": "openresty"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Content-Type": "application/json; charset=utf-8"
time=2023-02-15T15:58:43+08:00 level=debug msg= Request URL: "https://docsbuildworkeracrpubdev.azurecr.io/v2/loc-build/manifests/1.0.02230.181"
time=2023-02-15T15:58:43+08:00 level=debug msg= Request method: "GET"
time=2023-02-15T15:58:43+08:00 level=debug msg= Request headers:
time=2023-02-15T15:58:43+08:00 level=debug msg=   "User-Agent": "oras/0.16.0"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
time=2023-02-15T15:58:43+08:00 level=debug msg=   "Authorization": "*****"
time=2023-02-15T15:58:44+08:00 level=debug msg= Response Status: "200 OK"
time=2023-02-15T15:58:44+08:00 level=debug msg= Response headers:
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Docker-Content-Digest": "sha256:fac32d7727f87cb97f17f91a6cce7350933203f52f6979183ef7bbbbabd46b96"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Docker-Distribution-Api-Version": "registry/2.0"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Etag": "\"sha256:fac32d7727f87cb97f17f91a6cce7350933203f52f6979183ef7bbbbabd46b96\""
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Content-Type": "application/vnd.oci.image.manifest.v1+json"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Access-Control-Expose-Headers": "Docker-Content-Digest, WWW-Authenticate, Link, X-Ms-Correlation-Request-Id"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "X-Ms-Request-Id": "d706a174-51fa-4b70-95a8-8e9402c34a6d"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Server": "openresty"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Date": "Wed, 15 Feb 2023 07:58:44 GMT"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "X-Content-Type-Options": "nosniff"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Content-Length": "547"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Connection": "keep-alive"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "Strict-Transport-Security": "max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains"
time=2023-02-15T15:58:44+08:00 level=debug msg=   "X-Ms-Client-Request-Id": ""
time=2023-02-15T15:58:44+08:00 level=debug msg=   "X-Ms-Correlation-Request-Id": "8358d761-44df-4043-9226-796ab9f927c1"
Downloading fac32d7727f8 application/vnd.oci.image.manifest.v1+json
Processing  fac32d7727f8 application/vnd.oci.image.manifest.v1+json
Skipped     44136fa355b3 application/vnd.unknown.config.v1+json
Downloading 18f7f9143a9f loc-build
Downloaded  18f7f9143a9f loc-build
Downloaded  fac32d7727f8 application/vnd.oci.image.manifest.v1+json
Pulled docsbuildworkeracrpubdev.azurecr.io/loc-build:1.0.02230.181
Digest: sha256:fac32d7727f87cb97f17f91a6cce7350933203f52f6979183ef7bbbbabd46b96

After that, I can see one file under my temp folder, it is named in the following format.: oras_file_xxxxxxxxxx

What did you expect to happen?

  1. This temp file should be deleted after oras pull is completed.
  2. If this is by design, we can add one parameter to delete this temp file.

How can we reproduce it?

  1. Enable oras_cache
  2. Pull one image (it's one folder)

What is the version of your ORAS CLI?

Version: 0.16.0
Go version: go1.19.2
Git commit: d606fed
Git tree state: clean

What is your OS environment?

Windows

Are you willing to submit PRs to fix it?

  • Yes, I am willing to fix it.
@chi-qin chi-qin added the bug Something isn't working label Feb 15, 2023
@qweeah
Copy link
Contributor

qweeah commented Feb 15, 2023

Thanks @chi-qin for reporting! The file store used for pulling is not properly closed. @shizhMSFT Do we need to patch this fix to 0.15.X?

@shizhMSFT shizhMSFT added this to the v1.0.0-rc.2 milestone Feb 15, 2023
@shizhMSFT
Copy link
Contributor

@qweeah Yes, we do need to cherry pick and patch 0.15.x and 0.16.x until we release v1.0.0. /cc @FeynmanZhou

@shizhMSFT shizhMSFT added the good first issue Good for newcomers label Feb 15, 2023
@shizhMSFT
Copy link
Contributor

@qweeah Yes, we do need to cherry pick and patch 0.15.x and 0.16.x until we release v1.0.0. /cc @FeynmanZhou

Since 0.15.x and 0.16.x are v0 versions, users are encouraged to use v1.0.0 and tests its RC versions.

@chi-qin chi-qin mentioned this issue Feb 21, 2023
Merged
shizhMSFT pushed a commit that referenced this issue Feb 21, 2023
@chi-qin
fix: lazy close file store (#831)
1deed1c 
Fixes #815 
We should delete the temp file after pulling the image to reduce storage
usage

Signed-off-by: Chi Qin <chiqin@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chi-qin chi-qin

Labels
bug Something isn't working good first issue Good for newcomers
Projects
No open projects
ORAS-Planning
Status: Done
Milestone
v1.0.0-rc.2
Development

Successfully merging a pull request may close this issue.

fix: lazy close file store chi-qin/oras
3 participants
@qweeah @shizhMSFT @chi-qin