4.19.0-okd-scos.1 vSphere IPI Install Fails at Bootstrap

[yurivader]

hi,
I am currently in the process of installing
OKD 4.19.0-okd-scos.1
Platform: vsphere
Install method: IPI

cluster in a disconnected Environment.
Environment: Vsphere 8 vcenter
Sonatype Nexus as Registry Proxy
install-config.yaml.txt
install-config section for proxy;

imageDigestSources:
- mirrors:
  - sonartype-repo.local:11445
  source: quay.io
- mirrors:
  - sonartype-repo.local:11446
  source: registry.k8s.io
- mirrors:
  - sonartype-repo.local:11449
  source: registry.access.redhat.com
- mirrors:
  - sonartype-repo.local:11448
  source: registry.redhat.io

I do not have a direct internet connection from the cluster

on the Bootstrap node the Units
node-image-pull.service and
node-image-finish.service are marked as failed.

node-image-finish.service
fails due to dependency node-image-overlay.service
fails due to dependency node-image-pull.service

when looking at the logs for node-image-pull.service
it fails with error: --create specified but ref coreos/node-image already exists

May 22 06:55:35 localhost.localdomain node-image-pull.sh[23683]: Currently on CoreOS version 9.0.20250510-0
May 22 06:55:35 localhost.localdomain node-image-pull.sh[23683]: Target node image is quay.io/okd/scos-content@sha256:3813e6608a999756931d3d621932af9662>
May 22 06:55:35 localhost.localdomain node-image-pull.sh[23683]: Not in live environment
May 22 06:55:35 localhost.localdomain node-image-pull.sh[23683]: Pulling quay.io/okd/scos-content@sha256:3813e6608a999756931d3d621932af9662860e71a552b26[...]
May 22 06:55:35 localhost.localdomain ostree-containe[23829]: Fetching ostree-unverified-registry:quay.io/okd/scos-content@sha256:3813e6608a999756931d3d[...]
May 22 06:55:35 localhost.localdomain node-image-pull.sh[23829]: No changes in ostree-unverified-registry:quay.io/okd/scos-content@sha256:3813e6608a9997[...]
May 22 06:55:35 localhost.localdomain node-image-pull.sh[23887]: error: --create specified but ref coreos/node-image already exists
May 22 06:55:35 localhost.localdomain systemd[1]: node-image-pull.service: Main process exited, code=exited, status=1/FAILURE
May 22 06:55:35 localhost.localdomain systemd[1]: node-image-pull.service: Failed with result 'exit-code'.
May 22 06:55:35 localhost.localdomain systemd[1]: Failed to start Node Image Pull.

the openshift-installer shows following Log messages after the 15 min timeout:

INFO 2025-05-22 06:55:51.453820384 +0000 UTC m=+0.018762938 image pull 41a16d633e9ad10d435382e947d262b17349467b390d3f835e7db32981105d34 quay.io/okd/scos-release@sha256:26b3364692625d89573b65069f1353a75b7b5c4aa9eef76ac32bba89c14cc5f2
INFO 2025-05-22 06:55:51.558743754 +0000 UTC m=+0.123686312 container remove d972893a899c37e8220d620e4dd432d4cb32bf918c35acae12a48ea97b423db7 (image=quay.io/okd/scos-release@sha256:26b3364692625d89573b65069f1353a75b7b5c4aa9eef76ac32bba89c14cc5f2, name=flamboyant_mcclintock, io.openshift.release=4.19.0-okd-scos.1, io.openshift.release.base-image-digest=sha256:676522d547d87a6946a45b593171417861e637236e13feb339b1dc8430b9457e)
INFO error: unable to read image quay.io/okd/scos-release@sha256:26b3364692625d89573b65069f1353a75b7b5c4aa9eef76ac32bba89c14cc5f2: Get "https://quay.io/v2/": read tcp 10.123.231.40:38552->44.215.52.77:443: read: connection reset by peer

when pulling the Images mentioned via podman pull, they pull allright.

 [root@localhost ~]# podman pull quay.io/okd/scos-release@sha256:26b3364692625d89573b65069f1353a75b7b5c4aa9eef76ac32bba89c14cc5f2
Trying to pull quay.io/okd/scos-release@sha256:26b3364692625d89573b65069f1353a75b7b5c4aa9eef76ac32bba89c14cc5f2...
Getting image source signatures
Copying blob 58c18bd4e79f skipped: already exists
Copying blob fb6485c1d3bf skipped: already exists
Copying blob b0268b2d3348 skipped: already exists
Copying blob 2916c184a58b skipped: already exists
Copying blob 54cfc3a2e459 skipped: already exists
Copying blob 6b2459a9848d skipped: already exists
Copying config 41a16d633e done   |
Writing manifest to image destination
41a16d633e9ad10d435382e947d262b17349467b390d3f835e7db32981105d34

I 've tried different versions of OKD (4.18, 4.19.scos.0)
different cluster images, eg. scos-9.0.20250510-0-vmware.x86_64.ova and scos-9.0.20250515-0-vmware.x86_64.ova

Thank you all in advance for helping me solving this mystery!

[safodz]

Hi
from your logs it seems that it is not trying to reefer to your internal registry but to the public registry and this is because you used the public pull secret in you install coning fille and not the private pull registry pull secret

regards