Azure OKD installation failed (Image issue)

[mefistofeel]

Version

4.7.0-0.okd-2021-06-04-191031

Platform:

Azure

• IPI installation

What happened?

FCOS image URL is wrong

time="2021-06-09T14:20:01Z" level=debug msg="      Generating Image..."
time="2021-06-09T14:20:01Z" level=fatal msg="failed to fetch Metadata: failed to fetch dependency of \"Metadata\": failed to fetch dependency of \"Bootstrap Ignition Config\": failed to fetch dependency of \"Master Machines\": failed to generate asset \"Image\": testing/x86_64: No azure build found"
time="2021-06-09T14:23:24Z" level=debug msg="OpenShift Installer 4.7.0-0.okd-2021-06-04-191031"

Anything else we need to know?

Error in the Arch structure or in the fcos-stream.json

vendor/github.com/coreos/stream-metadata-go/stream/stream.go:27

// Arch contains release details for a particular hardware architecture
type Arch struct {
	Artifacts map[string]PlatformArtifacts `json:"artifacts"`
	Images    Images                       `json:"images,omitempty"`
	// RHELCoreOSExtensions is data specific to Red Hat Enterprise Linux CoreOS
	RHELCoreOSExtensions *rhcos.Extensions `json:"rhel-coreos-extensions,omitempty"`
}

File fcos-stream.json does not contain section rhel-coreos-extensions, which is present in file rhcos-stream.json

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[Eretuk]

Faced with the same issue. With the next installer: 4.8.0-0.okd-2021-10-10-030117
time="2021-10-11T12:44:13+02:00" level=debug msg=" Generating Image..." time="2021-10-11T12:44:13+02:00" level=fatal msg="failed to fetch Metadata: failed to fetch dependency of \"Metadata\": failed to fetch dependency of \"Bootstrap Ignition Config\": failed to fetch dependency of \"Master Machines\": failed to generate asset \"Image\": stable/x86_64: No azure build found"

[nwnt]

The same issue still persists with the latest release: 4.8.0-0.okd-2021-10-24-061736

[tthrone]

The same issue persists with release: 4.9.0-0.okd-2021-11-09-231731

[tthrone]

I did a bit of digging on this issue. The OP was correct with the issue in fcos-stream.json.
The issue is that FCOS is not published to a shared Azure Storage Account like the RHCOS image is.

Until FCOS does that, the rhel-coreos-extensions cannot be generated with a valid value.

To manually patch this value and get the 4.8 installer working (must have golang 1.14 or higher):

1. Clone https://github.com/vrutkovs/installer repository (required for OKD 4.8)
2. Checkout release-4.8-okd branch
3. Update hack/build.sh to add 'okd' to TAGS argument
4. Find Azure release url from data/data/fcos-stream.json and download it.
5. Untar VHD from download.
6. Create Azure Storage Account.
7. Upload VHD as page blob to a container under storage account.
8. Edit data/data/fcos-stream.json to add the json snippet below (add at the same level as "images")
9. Run ./hack/build.sh
10. Use the generated ./bin/openshift-install as you normally would.

** JSON to append to fcos-stream.json: **

"rhel-coreos-extensions": {
  "azure-disk": {
    "release": "34.20210626.3.10",
    "url": "https://[yourstorageaccount].blob.core.windows.net/fcos/fcos.vhd"
  }
}

A couple notes:

• That is the release at the time of this writing. That may change, so you would need to update the "release" property of the json snippet to match the release tied to Azure in your version of fcos-stream.json before putting the patch in.
• I largely suspect that you could put any url in there and entirely avoid having to download and upload to a storage account as the installer grabs a different os image and uploads it to the cluster storage account anyways. I would recommend trying it with a dummy url first. The upload to azure takes an hour or two.

[tthrone]

Side note: This fix should work for the 4.9 installer as well. You would use the official installer repo (https://github.com/openshift/installer/) and follow the same steps.

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closing this issue.

Details

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[alishchytovych]

/reopen

[openshift-ci]

@alishchytovych: You can't reopen an issue/PR unless you authored it or you are a collaborator.

Details

In response to this:

/reopen

The same issue with 4.9, post above doesn't help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.