registry mirrors #3221

Gsonovb · 2022-06-07T09:02:09Z

Summary

add the registry mirror with rewrite function

For example, k8s.gcr.io/pause:3.1 Redirect to registry.cn-hangzhou.aliyuncs.com/google_ containers/pause:3.1

Why is this important?

Because the network firewall, Google and other registries cannot be accessed, microk8s cannot be started normally.

The text was updated successfully, but these errors were encountered:

davix · 2022-06-07T10:07:01Z

Similar to #842?

neoaggelos · 2022-06-07T12:30:57Z

Hi @Gsonovb

This is already possible, you can configure containerd like this:

mkdir /var/snap/microk8s/current/args/certs.d/k8s.gcr.io
echo '
server = "https://k8s.gcr.io"

[host."https://registry.cn-hangzhou.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]
' | sudo tee -a /var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

Though there is relevant documentation in https://microk8s.io/docs/dockerhub-limits, I think we could use a section regarding the k8s.gcr.io registry mirrors as well.

Afterwards, restart containerd with:

sudo snap restart microk8s

Gsonovb · 2022-06-08T00:48:40Z

HI @davix
yes. I recently found such a configuration file when testing k3s, which works well.

Hi @neoaggelos
I've seen that document, but it doesn't work here.
Use microk8s CTR I pull k8s gcr. Io/kube-controller-manager:v1.24.0 --http dump, request k8s gcr. IO then timeout

#/var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

server = "https://registry.cn-hangzhou.aliyuncs.com/google_containers"

[host."registry.cn-hangzhou.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]

neoaggelos · 2022-06-08T01:42:56Z

HI @davix
yes. I recently found such a configuration file when testing k3s, which works well.

Hi @neoaggelos
I've seen that document, but it doesn't work here.
Use microk8s CTR I pull k8s gcr. Io/kube-controller-manager:v1.24.0 --http dump, request k8s gcr. IO then timeout
#/var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

server = "https://registry.cn-hangzhou.aliyuncs.com/google_containers"

[host."registry.cn-hangzhou.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]

For the microk8s.ctr command, you must also include --hosts-dir /var/snap/microk8s/current/args/certs.d in your command line, otherwise the host overrides are ignored.

Did you try creating a deployment?

neoaggelos · 2022-06-29T08:59:11Z

The process of setting up a registry mirror is now documented in https://microk8s.io/docs/registry-private#configure-registry-mirrors-7. Closing the issue, please re-open if the documented steps are failing. Thank you!

CoachRun · 2022-09-19T02:50:20Z

Hi @Gsonovb

This is already possible, you can configure containerd like this:
mkdir /var/snap/microk8s/current/args/certs.d/k8s.gcr.io
echo '
server = "https://k8s.gcr.io"

[host."https://registry.cn-hangzhou.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]
' | sudo tee -a /var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml
Though there is relevant documentation in https://microk8s.io/docs/dockerhub-limits, I think we could use a section regarding the registry mirrors as well.k8s.gcr.io

Afterwards, restart containerd with:
sudo snap restart microk8s

MicroK8s v1.25.0 revision 3883
Try step by step, not working.

CoachRun · 2022-09-19T02:59:06Z

If I use this scheme, I get the problem

echo '
server = "https://k8s.gcr.io"

[host."https://registry.cn-hangzhou.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]
' | sudo tee -a /var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

got an event:

Failed to create pod sandbox: rpc error: code = DeadlineExceeded desc = failed to get sandbox image "k8s.gcr.io/pause:3.7": failed to pull image "k8s.gcr.io/pause:3.7": failed to pull and unpack image "k8s.gcr.io/pause:3.7": failed to resolve reference "k8s.gcr.io/pause:3.7": failed to do request: Head "https://k8s.gcr.io/v2/pause/manifests/3.7": dial tcp 108.177.97.82:443: i/o timeout

If I use this scheme, I get the problem

echo '
server = "https://registry.cn-hangzhou.aliyuncs.com/google_containers"

[host."https://registry.cn-hangzhou.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]
' | sudo tee -a /var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

got an event:

  Warning  FailedCreatePodSandBox  4m5s (x26 over 9m29s)  kubelet  Failed to create pod sandbox: rpc error: code = NotFound desc = failed to get sandbox image "k8s.gcr.io/pause:3.7": failed to pull image "k8s.gcr.io/pause:3.7": failed to pull and unpack image "k8s.gcr.io/pause:3.7": failed to resolve reference "k8s.gcr.io/pause:3.7": k8s.gcr.io/pause:3.7: not found

neoaggelos · 2022-09-19T23:21:47Z

Hi @CoachRun,

Looks like the server has changed since the documentation was last updated. Can you try this instead?

echo '
server = "https://k8s.gcr.io"

[host."https://registry.aliyuncs.com/v2/google_containers"]
  capabilities = ["pull", "resolve"]
  override_path = true
' | sudo tee -a /var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

CoachRun · 2022-09-20T01:34:46Z

Hi @CoachRun,

Looks like the server has changed since the documentation was last updated. Can you try this instead?
echo '
server = "https://k8s.gcr.io"

[host."https://registry.aliyuncs.com/v2/google_containers"]
  capabilities = ["pull", "resolve"]
  override_path = true
' | sudo tee -a /var/snap/microk8s/current/args/certs.d/k8s.gcr.io/hosts.toml

I gave it a try, not working

Warning  FailedCreatePodSandBox  13s (x10 over 2m8s)    kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't copy bootstrap data to pipe: write init-p: broken pipe: unknown

neoaggelos · 2022-09-20T06:17:54Z

broken pipe: unknown

That is most interesting. Does a sudo snap restart microk8s help? If not, can you try the following and paste the output here?

microk8s ctr image pull k8s.gcr.io/pause:3.7 --hosts-dir /var/snap/microk8s/current/args/certs.d
microk8s ctr image pull registry.aliyuncs.com/google_containers/pause:3.7

Both commands should work. If not, append --http-dump to further investigate why the image pull is failing.

CoachRun · 2022-09-20T09:36:09Z

uh...., It is worked.

without --http-dump flag

admin@test:~$ microk8s ctr image pull k8s.gcr.io/pause:3.7 --hosts-dir /var/snap/microk8s/current/args/certs.d
k8s.gcr.io/pause:3.7:                                                             resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c:    exists         |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:f81611a21cf91214c1ea751c5b525931a0e2ebabe62b3937b6158039ff6f922d: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7582c2cc65ef30105b84c1c6812f71c8012663c6352b01fe2f483238313ab0ed:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:221177c6082a88ea4f6240ab2450d540955ac6f4d5454f0e15751b653ebda165:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.3 s                                                                    total:   0.0 B (0.0 B/s)
unpacking linux/amd64 sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c...
done: 9.857023ms
admin@test:~$ microk8s ctr image pull registry.aliyuncs.com/google_containers/pause:3.7
registry.aliyuncs.com/google_containers/pause:3.7:                                resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:f81611a21cf91214c1ea751c5b525931a0e2ebabe62b3937b6158039ff6f922d: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7582c2cc65ef30105b84c1c6812f71c8012663c6352b01fe2f483238313ab0ed:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:221177c6082a88ea4f6240ab2450d540955ac6f4d5454f0e15751b653ebda165:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.2 s                                                                    total:   0.0 B (0.0 B/s)
unpacking linux/amd64 sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c...
done: 10.731241ms

and --http-dump flag

admin@test:~$ microk8s ctr image pull k8s.gcr.io/pause:3.7 --hosts-dir /var/snap/microk8s/current/args/certs.d --http-dump
INFO[0000] HEAD /v2/google_containers/pause/manifests/3.7?ns=k8s.gcr.io HTTP/1.1
INFO[0000] Host: registry.aliyuncs.com
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] User-Agent: containerd/v1.6.6
INFO[0000]
INFO[0000] HTTP/1.1 401 Unauthorized
INFO[0000] Content-Length: 166
INFO[0000] Content-Type: application/json; charset=utf-8
INFO[0000] Date: Tue, 20 Sep 2022 09:33:42 GMT
INFO[0000] Docker-Distribution-Api-Version: registry/2.0
INFO[0000] Www-Authenticate: Bearer realm="https://dockerauth.cn-hangzhou.aliyuncs.com/auth",service="registry.aliyuncs.com:cn-hangzhou:26842",scope="repository:google_containers/pause:pull"
INFO[0000]
INFO[0000] GET /auth?scope=repository%3Agoogle_containers%2Fpause%3Apull&scope=repository%3Apause%3Apull&service=registry.aliyuncs.com%3Acn-hangzhou%3A26842 HTTP/1.1
INFO[0000] Host: dockerauth.cn-hangzhou.aliyuncs.com
INFO[0000] User-Agent: containerd/v1.6.6
INFO[0000]
INFO[0000] HTTP/1.1 200 OK
INFO[0000] Transfer-Encoding: chunked
INFO[0000] Cache-Control: no-cache, no-store, max-age=0, must-revalidate
INFO[0000] Connection: keep-alive
INFO[0000] Content-Type: application/json;charset=UTF-8
INFO[0000] Date: Tue, 20 Sep 2022 09:33:42 GMT
INFO[0000] Eagleeye-Traceid: 0bc3b4b216636664221892199e740d
INFO[0000] Expires: 0
INFO[0000] Pragma: no-cache
INFO[0000] Server: Tengine
INFO[0000] Set-Cookie: XSRF-TOKEN=63ea740b-4e4d-45f2-a393-9614a8c708d3; Path=/; HttpOnly
INFO[0000] Strict-Transport-Security: max-age=31536000
INFO[0000] Timing-Allow-Origin: *
INFO[0000] Vary: Accept-Encoding
INFO[0000] X-Content-Type-Options: nosniff
INFO[0000] X-Frame-Options: DENY
INFO[0000] X-Xss-Protection: 1; mode=block
INFO[0000]
INFO[0000] 67a
INFO[0000] {"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6InJlZ2lzdHJ5LmFsaXl1bmNzLmNvbTpjbi1oYW5nemhvdToyNjg0MiIsInN1YiI6IiIsImlhdCI6MTY2MzY2NjQyMiwianRpIjoiWFFLWkY3SmJSOW1zQTlDNFV3dkIzdyIsIm5iZiI6MTY2MzY2NjEyMiwiZXhwIjoxNjYzNjY3MDIyLCJhY2Nlc3MiOlt7Im5hbWUiOiJnb29nbGVfY29udGFpbmVycy9wYXVzZSIsInR5cGUiOiJyZXBvc2l0b3J5IiwiYWN0aW9ucyI6WyJwdWxsIl19XX0.cOng-Mg39zH8SragWXbLvGcmtz_ARMYp1jzGVRpXWbt3AcqksVK2J-VkBI4qIL6TZ3QPnT6ok1I1Dql2iu8Nri7L_AgbYB5K-6gSJ7P-nG1VWw_R48uGK1Bn-LcTRJOZZbVNAYibXbwROjtVbuLmkwFLyU5AjtC9nlh9pO6Z5jbBJcxk_VasD9GnCLHnM2GqsMSvhn1WNcfNuKsi97-VhPJgBIlQueN9jNnNAvk0nClrZFVBG8Dp9-lye91JOmMuuAFboPDBjGOH8fHW_688JQphsM-eyGmp4q9oO7Rt5tPUeB0o8508U3ZvD_yEv5s_Fc8t3jf5ZtNjX5AhmEgxmA","token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6InJlZ2lzdHJ5LmFsaXl1bmNzLmNvbTpjbi1oYW5nemhvdToyNjg0MiIsInN1YiI6IiIsImlhdCI6MTY2MzY2NjQyMiwianRpIjoiWFFLWkY3SmJSOW1zQTlDNFV3dkIzdyIsIm5iZiI6MTY2MzY2NjEyMiwiZXhwIjoxNjYzNjY3MDIyLCJhY2Nlc3MiOlt7Im5hbWUiOiJnb29nbGVfY29udGFpbmVycy9wYXVzZSIsInR5cGUiOiJyZXBvc2l0b3J5IiwiYWN0aW9ucyI6WyJwdWxsIl19XX0.cOng-Mg39zH8SragWXbLvGcmtz_ARMYp1jzGVRpXWbt3AcqksVK2J-VkBI4qIL6TZ3QPnT6ok1I1Dql2iu8Nri7L_AgbYB5K-6gSJ7P-nG1VWw_R48uGK1Bn-LcTRJOZZbVNAYibXbwROjtVbuLmkwFLyU5AjtC9nlh9pO6Z5jbBJcxk_VasD9GnCLHnM2GqsMSvhn1WNcfNuKsi97-VhPJgBIlQueN9jNnNAvk0nClrZFVBG8Dp9-lye91JOmMuuAFboPDBjGOH8fHW_688JQphsM-eyGmp4q9oO7Rt5tPUeB0o8508U3ZvD_yEv5s_Fc8t3jf5ZtNjX5AhmEgxmA"}
INFO[0000] 0
INFO[0000]
INFO[0000] HEAD /v2/google_containers/pause/manifests/3.7?ns=k8s.gcr.io HTTP/1.1
INFO[0000] Host: registry.aliyuncs.com
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6InJlZ2lzdHJ5LmFsaXl1bmNzLmNvbTpjbi1oYW5nemhvdToyNjg0MiIsInN1YiI6IiIsImlhdCI6MTY2MzY2NjQyMiwianRpIjoiWFFLWkY3SmJSOW1zQTlDNFV3dkIzdyIsIm5iZiI6MTY2MzY2NjEyMiwiZXhwIjoxNjYzNjY3MDIyLCJhY2Nlc3MiOlt7Im5hbWUiOiJnb29nbGVfY29udGFpbmVycy9wYXVzZSIsInR5cGUiOiJyZXBvc2l0b3J5IiwiYWN0aW9ucyI6WyJwdWxsIl19XX0.cOng-Mg39zH8SragWXbLvGcmtz_ARMYp1jzGVRpXWbt3AcqksVK2J-VkBI4qIL6TZ3QPnT6ok1I1Dql2iu8Nri7L_AgbYB5K-6gSJ7P-nG1VWw_R48uGK1Bn-LcTRJOZZbVNAYibXbwROjtVbuLmkwFLyU5AjtC9nlh9pO6Z5jbBJcxk_VasD9GnCLHnM2GqsMSvhn1WNcfNuKsi97-VhPJgBIlQueN9jNnNAvk0nClrZFVBG8Dp9-lye91JOmMuuAFboPDBjGOH8fHW_688JQphsM-eyGmp4q9oO7Rt5tPUeB0o8508U3ZvD_yEv5s_Fc8t3jf5ZtNjX5AhmEgxmA
INFO[0000] User-Agent: containerd/v1.6.6
INFO[0000]
INFO[0000] HTTP/1.1 200 OK
INFO[0000] Content-Length: 2761
INFO[0000] Content-Type: application/vnd.docker.distribution.manifest.list.v2+json
INFO[0000] Date: Tue, 20 Sep 2022 09:33:42 GMT
INFO[0000] Docker-Content-Digest: sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c
INFO[0000] Docker-Distribution-Api-Version: registry/2.0
INFO[0000] Etag: "sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c"
INFO[0000]
k8s.gcr.io/pause:3.7:                                                             resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:f81611a21cf91214c1ea751c5b525931a0e2ebabe62b3937b6158039ff6f922d: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7582c2cc65ef30105b84c1c6812f71c8012663c6352b01fe2f483238313ab0ed:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:221177c6082a88ea4f6240ab2450d540955ac6f4d5454f0e15751b653ebda165:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.2 s                                                                    total:   0.0 B (0.0 B/s)
unpacking linux/amd64 sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c...
done: 6.900422ms
admin@test:~$ microk8s ctr image pull registry.aliyuncs.com/google_containers/pause:3.7 --http-dump
INFO[0000] HEAD /v2/google_containers/pause/manifests/3.7 HTTP/1.1
INFO[0000] Host: registry.aliyuncs.com
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] User-Agent: containerd/v1.6.6
INFO[0000]
INFO[0000] HTTP/1.1 401 Unauthorized
INFO[0000] Content-Length: 166
INFO[0000] Content-Type: application/json; charset=utf-8
INFO[0000] Date: Tue, 20 Sep 2022 09:34:03 GMT
INFO[0000] Docker-Distribution-Api-Version: registry/2.0
INFO[0000] Www-Authenticate: Bearer realm="https://dockerauth.cn-hangzhou.aliyuncs.com/auth",service="registry.aliyuncs.com:cn-hangzhou:26842",scope="repository:google_containers/pause:pull"
INFO[0000]
INFO[0000] GET /auth?scope=repository%3Agoogle_containers%2Fpause%3Apull&service=registry.aliyuncs.com%3Acn-hangzhou%3A26842 HTTP/1.1
INFO[0000] Host: dockerauth.cn-hangzhou.aliyuncs.com
INFO[0000] User-Agent: containerd/v1.6.6
INFO[0000]
INFO[0000] HTTP/1.1 200 OK
INFO[0000] Transfer-Encoding: chunked
INFO[0000] Cache-Control: no-cache, no-store, max-age=0, must-revalidate
INFO[0000] Connection: keep-alive
INFO[0000] Content-Type: application/json;charset=UTF-8
INFO[0000] Date: Tue, 20 Sep 2022 09:34:03 GMT
INFO[0000] Eagleeye-Traceid: 0bc3b4ab16636664434543138e17d1
INFO[0000] Expires: 0
INFO[0000] Pragma: no-cache
INFO[0000] Server: Tengine
INFO[0000] Set-Cookie: XSRF-TOKEN=312f11da-c996-4ef6-8433-b4cfae6928a3; Path=/; HttpOnly
INFO[0000] Strict-Transport-Security: max-age=31536000
INFO[0000] Timing-Allow-Origin: *
INFO[0000] Vary: Accept-Encoding
INFO[0000] X-Content-Type-Options: nosniff
INFO[0000] X-Frame-Options: DENY
INFO[0000] X-Xss-Protection: 1; mode=block
INFO[0000]
INFO[0000] 67a
INFO[0000] {"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6InJlZ2lzdHJ5LmFsaXl1bmNzLmNvbTpjbi1oYW5nemhvdToyNjg0MiIsInN1YiI6IiIsImlhdCI6MTY2MzY2NjQ0MywianRpIjoieGJSbG1yQWRZV3JZQmxieGdJeExrUSIsIm5iZiI6MTY2MzY2NjE0MywiZXhwIjoxNjYzNjY3MDQzLCJhY2Nlc3MiOlt7Im5hbWUiOiJnb29nbGVfY29udGFpbmVycy9wYXVzZSIsInR5cGUiOiJyZXBvc2l0b3J5IiwiYWN0aW9ucyI6WyJwdWxsIl19XX0.nPUhpKczs5zAxtChHc0oFnALYisbIK2vBlJY2W2XXd43utYL1PcZo-s0ol0OeQ99UuxlYs5nnP_ysIQR_4mhiDBac0-jVpUJd9uOAWwvfi-a2avmvmnSCsdVsIDz8bYsBzUdgcSD35o4zh18eFGzo8mMEJHjnZ-oEWN13JmoiPe5RU9xcyrUwQHfhDX3Xo67jcVzLzTtHohcQYxjlihIeG0tV1FM1OmynyFyUekzWSMIKScu2TLMcqcJFrDzO6RsWiYslAuZDTOVzoKJIoeFGQnJQnckWPJDQxNOyUKrTvS2vcImHJzxJDUcv8NA7e3olHeUJ2Qv6VNTUNGxdBucKA","token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6InJlZ2lzdHJ5LmFsaXl1bmNzLmNvbTpjbi1oYW5nemhvdToyNjg0MiIsInN1YiI6IiIsImlhdCI6MTY2MzY2NjQ0MywianRpIjoieGJSbG1yQWRZV3JZQmxieGdJeExrUSIsIm5iZiI6MTY2MzY2NjE0MywiZXhwIjoxNjYzNjY3MDQzLCJhY2Nlc3MiOlt7Im5hbWUiOiJnb29nbGVfY29udGFpbmVycy9wYXVzZSIsInR5cGUiOiJyZXBvc2l0b3J5IiwiYWN0aW9ucyI6WyJwdWxsIl19XX0.nPUhpKczs5zAxtChHc0oFnALYisbIK2vBlJY2W2XXd43utYL1PcZo-s0ol0OeQ99UuxlYs5nnP_ysIQR_4mhiDBac0-jVpUJd9uOAWwvfi-a2avmvmnSCsdVsIDz8bYsBzUdgcSD35o4zh18eFGzo8mMEJHjnZ-oEWN13JmoiPe5RU9xcyrUwQHfhDX3Xo67jcVzLzTtHohcQYxjlihIeG0tV1FM1OmynyFyUekzWSMIKScu2TLMcqcJFrDzO6RsWiYslAuZDTOVzoKJIoeFGQnJQnckWPJDQxNOyUKrTvS2vcImHJzxJDUcv8NA7e3olHeUJ2Qv6VNTUNGxdBucKA"}
INFO[0000] 0
INFO[0000]
INFO[0000] HEAD /v2/google_containers/pause/manifests/3.7 HTTP/1.1
INFO[0000] Host: registry.aliyuncs.com
INFO[0000] Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*
INFO[0000] Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6InJlZ2lzdHJ5LmFsaXl1bmNzLmNvbTpjbi1oYW5nemhvdToyNjg0MiIsInN1YiI6IiIsImlhdCI6MTY2MzY2NjQ0MywianRpIjoieGJSbG1yQWRZV3JZQmxieGdJeExrUSIsIm5iZiI6MTY2MzY2NjE0MywiZXhwIjoxNjYzNjY3MDQzLCJhY2Nlc3MiOlt7Im5hbWUiOiJnb29nbGVfY29udGFpbmVycy9wYXVzZSIsInR5cGUiOiJyZXBvc2l0b3J5IiwiYWN0aW9ucyI6WyJwdWxsIl19XX0.nPUhpKczs5zAxtChHc0oFnALYisbIK2vBlJY2W2XXd43utYL1PcZo-s0ol0OeQ99UuxlYs5nnP_ysIQR_4mhiDBac0-jVpUJd9uOAWwvfi-a2avmvmnSCsdVsIDz8bYsBzUdgcSD35o4zh18eFGzo8mMEJHjnZ-oEWN13JmoiPe5RU9xcyrUwQHfhDX3Xo67jcVzLzTtHohcQYxjlihIeG0tV1FM1OmynyFyUekzWSMIKScu2TLMcqcJFrDzO6RsWiYslAuZDTOVzoKJIoeFGQnJQnckWPJDQxNOyUKrTvS2vcImHJzxJDUcv8NA7e3olHeUJ2Qv6VNTUNGxdBucKA
INFO[0000] User-Agent: containerd/v1.6.6
INFO[0000]
INFO[0000] HTTP/1.1 200 OK
INFO[0000] Content-Length: 2761
INFO[0000] Content-Type: application/vnd.docker.distribution.manifest.list.v2+json
INFO[0000] Date: Tue, 20 Sep 2022 09:34:03 GMT
INFO[0000] Docker-Content-Digest: sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c
INFO[0000] Docker-Distribution-Api-Version: registry/2.0
INFO[0000] Etag: "sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c"
INFO[0000]
registry.aliyuncs.com/google_containers/pause:3.7:                                resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:f81611a21cf91214c1ea751c5b525931a0e2ebabe62b3937b6158039ff6f922d: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:7582c2cc65ef30105b84c1c6812f71c8012663c6352b01fe2f483238313ab0ed:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:221177c6082a88ea4f6240ab2450d540955ac6f4d5454f0e15751b653ebda165:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.2 s                                                                    total:   0.0 B (0.0 B/s)
unpacking linux/amd64 sha256:bb6ed397957e9ca7c65ada0db5c5d1c707c9c8afc80a94acbe69f3ae76988f0c...
done: 10.02566ms

neoaggelos · 2022-09-20T10:44:45Z

Great, the --http-dump flag is just for debugging purposes. I suppose the cluster should now be working properly as well, is it?

The broken pipe error could have been something transient. Thanks for your help debugging this @CoachRun, I'm going to update the documentation accordingly

CoachRun · 2022-09-20T15:02:56Z

Thanks for solve networking problem. It is work.

But node status is Not Ready. The pipe problem remains. I don't know how to solve it.

admin@test:~$ microk8s kubectl get nodes -o wide
NAME                   STATUS     ROLES    AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
test.minemicrok8s.io   NotReady   <none>   13h   v1.25.0   172.17.8.101   <none>        Ubuntu 20.04.1 LTS   5.4.0-47-generic   containerd://1.6.6

Warning  FailedCreatePodSandBox  79s (x26 over 6m59s)  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't copy bootstrap data to pipe: write init-p: broken pipe: unknown

neoaggelos mentioned this issue Jun 29, 2022

MicroK8s dashboard not working- CentOS #3188

Closed

neoaggelos closed this as completed Jun 29, 2022

javad87 mentioned this issue Nov 7, 2022

2 single node microk8s on different servers suddenly do not create pods after working perfectly for months... #3545

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

registry mirrors #3221

registry mirrors #3221

Gsonovb commented Jun 7, 2022

davix commented Jun 7, 2022

neoaggelos commented Jun 7, 2022 •

edited

Gsonovb commented Jun 8, 2022

neoaggelos commented Jun 8, 2022

neoaggelos commented Jun 29, 2022

CoachRun commented Sep 19, 2022

CoachRun commented Sep 19, 2022

neoaggelos commented Sep 19, 2022 •

edited

CoachRun commented Sep 20, 2022

neoaggelos commented Sep 20, 2022

CoachRun commented Sep 20, 2022

neoaggelos commented Sep 20, 2022

CoachRun commented Sep 20, 2022

registry mirrors #3221

registry mirrors #3221

Comments

Gsonovb commented Jun 7, 2022

Summary

Why is this important?

davix commented Jun 7, 2022

neoaggelos commented Jun 7, 2022 • edited

Gsonovb commented Jun 8, 2022

neoaggelos commented Jun 8, 2022

neoaggelos commented Jun 29, 2022

CoachRun commented Sep 19, 2022

CoachRun commented Sep 19, 2022

neoaggelos commented Sep 19, 2022 • edited

CoachRun commented Sep 20, 2022

neoaggelos commented Sep 20, 2022

CoachRun commented Sep 20, 2022

neoaggelos commented Sep 20, 2022

CoachRun commented Sep 20, 2022

neoaggelos commented Jun 7, 2022 •

edited

neoaggelos commented Sep 19, 2022 •

edited