Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

socket leaky on GetBlob if server response code not StatusOK #1177

Closed
wu0407 opened this issue Mar 14, 2021 · 2 comments · Fixed by #1178
Closed

socket leaky on GetBlob if server response code not StatusOK #1177

wu0407 opened this issue Mar 14, 2021 · 2 comments · Fixed by #1178

Comments

@wu0407
Copy link
Contributor

wu0407 commented Mar 14, 2021

if use dockerImageSource GetBlob to get blob, and registry server response not StatusOK. That tcp connection state is CLOSE_WAIT, so tcp socket leaked. may be cause dial tcp 10.19.0.3:443: socket: too many open files .

The error like Error fetching blob: invalid status code from registry 404 (Not Found)

#lsof -p  318050 -n -P

image-tra 318050 root *838u     IPv4 242209834       0t0       TCP 10.19.0.22:60270->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *839u     IPv4 242205027       0t0       TCP 10.19.0.22:60268->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *840u     IPv4 242192365       0t0       TCP 10.19.0.22:60412->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *841u     IPv4 242206891       0t0       TCP 10.19.0.22:60394->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *842u     IPv4 242196307       0t0       TCP 10.19.0.22:60406->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *843u     IPv4 242192364       0t0       TCP 10.19.0.22:60408->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *844u     IPv4 242197478       0t0       TCP 10.19.0.22:60524->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *845u     IPv4 242206909       0t0       TCP 10.19.0.22:60604->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *846u     IPv4 242201116       0t0       TCP 10.19.0.22:60696->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *847u     IPv4 242202286       0t0       TCP 10.19.0.22:60678->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *848u     IPv4 242208813       0t0       TCP 10.19.0.22:60600->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *849u     IPv4 242201114       0t0       TCP 10.19.0.22:60682->169.254.0.42:443 (CLOSE_WAIT)
image-tra 318050 root *850u     IPv4 242206925       0t0       TCP 10.19.0.22:60792->169.254.0.42:443 (ESTABLISHED)
image-tra 318050 root *851u     IPv4 242202292       0t0       TCP 10.19.0.22:60740->169.254.0.42:443 (ESTABLISHED)
image-tra 318050 root *852u     IPv4 242210821       0t0       TCP 10.19.0.22:60800->169.254.0.42:443 (ESTABLISHED)
image-tra 318050 root *853u     sock       0,8       0t0 242210824 protocol: TCP
image-tra 318050 root *854u     sock       0,8       0t0 242210826 protocol: TCP
image-tra 318050 root *855u     sock       0,8       0t0 242197493 protocol: TCP
image-tra 318050 root *856u     sock       0,8       0t0 242208853 protocol: TCP
image-tra 318050 root *857u     sock       0,8       0t0 242197495 protocol: TCP
image-tra 318050 root *858u     sock       0,8       0t0 242208854 protocol: TCP
wu0407 added a commit to wu0407/image that referenced this issue Mar 14, 2021
@wu0407
fix GetBlob socket leak
490e13c 
Closes containers#1177
This was referenced Mar 14, 2021
Merged
Merged
vrothberg pushed a commit that referenced this issue Mar 15, 2021
@wu0407 @vrothberg
fix GetBlob socket leak
71dd988 
Closes #1177
@mtrmac
Copy link
Collaborator

mtrmac commented Mar 15, 2021

Also, AFAICS:

  • docker.GetDigest (even on success)
  • docker.makeRequestToResolvedURL
  • docker.getExternalBlob

wu0407 added a commit to wu0407/image that referenced this issue Mar 16, 2021
@wu0407
fix GetBlob socket leak
f37a277 
Closes containers#1177
@wu0407
Copy link
Contributor Author

wu0407 commented Mar 17, 2021

Also, AFAICS:

  • docker.GetDigest (even on success)
  • docker.makeRequestToResolvedURL
  • docker.getExternalBlob

push the fix #1183

lsm5 added a commit to lsm5/containers-image that referenced this issue Apr 9, 2021
@lsm5
v5.11.0
1f072d1 
* Enable subdomain matching in registries.conf
* registries.conf: configure credential helpers
* Bump github.com/containers/storage from 1.28.0 to 1.28.1
* Bump github.com/klauspost/compress from 1.11.12 to 1.11.13
* Bump github.com/vbauerster/mpb/v6 from 6.0.2 to 6.0.3
* storage destination: improve doc comments re: concurrency
* Spelling
* Bump github.com/containers/storage from 1.27.0 to 1.28.0
* manifest: GuessMIMEType: support OCI artifacts
* copy: early commit of storage blobs
* fix docker.GetDigest docker.makeRequestToResolvedURL docker.getExternalBlob socket leak
* fix typo in docs/containers-registries.conf.d.5.md
* fix GetBlob socket leak Closes containers#1177
* platform matcher: use compat matrix
* Cirrus: Migrate from Travis CI
* add option to download foreign layers contents
* Bump github.com/klauspost/compress from 1.11.9 to 1.11.12
* storageImageSource.LayerInfosForCopy(): return uncompressed MediaTypes
* storage: GetBlob: write to a local tempfile
* Bump github.com/klauspost/compress from 1.11.7 to 1.11.9
* Bump github.com/ulikunitz/xz from 0.5.9 to 0.5.10
* Bump github.com/containers/storage from 1.26.0 to 1.27.0
* shortnames: correctly determine cache directory for rootless
* README: godoc: point to v5
* Check destination image exists copy optimization
* copy: compute blob compression on reused blobs based on source MediaType
* copy: provide compression info about copied blobs
* shortnames: resolution error: point to manpage if config is absent
* bump to v5.11.0-dev
* short-name-aliases.conf: use cache folders instead of $HOME
* update progress-bar lib mpb to v6
* Bump github.com/containers/storage from 1.24.5 to 1.25.0
* Bump github.com/containers/ocicrypt from 1.0.3 to 1.1.0
* Update containers-registries.conf.5.md
* Fix writing signatures for updated multi-arch image components
* Beautify copyBlobStream a bit
* move to v5.10.2-dev

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
lsm5 added a commit that referenced this issue Apr 9, 2021
@lsm5
v5.11.0
f943351 
* Enable subdomain matching in registries.conf
* registries.conf: configure credential helpers
* Bump github.com/containers/storage from 1.28.0 to 1.28.1
* Bump github.com/klauspost/compress from 1.11.12 to 1.11.13
* Bump github.com/vbauerster/mpb/v6 from 6.0.2 to 6.0.3
* storage destination: improve doc comments re: concurrency
* Spelling
* Bump github.com/containers/storage from 1.27.0 to 1.28.0
* manifest: GuessMIMEType: support OCI artifacts
* copy: early commit of storage blobs
* fix docker.GetDigest docker.makeRequestToResolvedURL docker.getExternalBlob socket leak
* fix typo in docs/containers-registries.conf.d.5.md
* fix GetBlob socket leak Closes #1177
* platform matcher: use compat matrix
* Cirrus: Migrate from Travis CI
* add option to download foreign layers contents
* Bump github.com/klauspost/compress from 1.11.9 to 1.11.12
* storageImageSource.LayerInfosForCopy(): return uncompressed MediaTypes
* storage: GetBlob: write to a local tempfile
* Bump github.com/klauspost/compress from 1.11.7 to 1.11.9
* Bump github.com/ulikunitz/xz from 0.5.9 to 0.5.10
* Bump github.com/containers/storage from 1.26.0 to 1.27.0
* shortnames: correctly determine cache directory for rootless
* README: godoc: point to v5
* Check destination image exists copy optimization
* copy: compute blob compression on reused blobs based on source MediaType
* copy: provide compression info about copied blobs
* shortnames: resolution error: point to manpage if config is absent
* bump to v5.11.0-dev
* short-name-aliases.conf: use cache folders instead of $HOME
* update progress-bar lib mpb to v6
* Bump github.com/containers/storage from 1.24.5 to 1.25.0
* Bump github.com/containers/ocicrypt from 1.0.3 to 1.1.0
* Update containers-registries.conf.5.md
* Fix writing signatures for updated multi-arch image components
* Beautify copyBlobStream a bit
* move to v5.10.2-dev

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix GetBlob socket leak wu0407/image
2 participants
@mtrmac @wu0407