Cluster unable to update in 4.15.0-0.okd-2024-02-10-035534

[llomgui]

Hello,

I tried to update a cluster to the latest version 4.15.0-0.okd-2024-02-10-035534.
It looks like rpm-ostree does not want to use proxy variables set in /etc/mco/proxy.env.

The master node keeps failing on rpm-ostree rebase --experimental ostree-unverified-registry:quay.io/openshift/okd-content@sha256:114f56ffe30b010ee62acc1744e195954e7194cc816b8fc0a23802fa048a50df > Timeout issue on quay.io.
It is not using the proxy as it should.

I also tried the following without luck:
mkdir -p /etc/systemd/system/rpm-ostreed.service.d cat > /etc/systemd/system/rpm-ostreed.service.d/http-proxy.conf << EOF [Service] Environment="http_proxy=my-proxy:8080" EOF

The following commands work:

export HTTP_PROXY=my-proxy:8080 podman pull quay.io/openshift/okd-content@sha256:114f56ffe30b010ee62acc1744e195954e7194cc816b8fc0a23802fa048a50df

[vrutkovs]

Please attach (or upload to the public file sharing service) must-gather archive

[llomgui]

I created the same issue 1 year ago: #1481
But this time the solution is not working.

[nate-duke]

@llomgui you can try using the redhatter's behind-the-scenes tool o-must-gather. I've not used this myself but if your company doesn't allow you to share logs with the community then that may be your best bet.

[llomgui]

Thank you @nate-duke If Vadim forces me I will.

But I found something: ostreedev/ostree-rs-ext#582

It looks like they reset all env variable in the latest ostree version. It might be my issue here @vrutkovs

[llomgui]

And their hack creating a file /usr/local/bin/skopeo works 👍

[vrutkovs]

Lets track this in #1481

[tonus-sebastian]

anyone give how to step by step upgrade cluster

[llomgui]

See my comment: #1481 (comment)