Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for IBM Cloud #277

Closed
5 tasks done
ashcrow opened this issue Sep 16, 2019 · 30 comments
Closed
5 tasks done

Add support for IBM Cloud #277

ashcrow opened this issue Sep 16, 2019 · 30 comments
Assignees
@dustymabe
Labels
area/platforms cloud* related to public/private clouds jira for syncing to jira

Comments

@ashcrow
Copy link
Member

ashcrow commented Sep 16, 2019 •
edited

In order to properly support this, we should:

  • reserve and document a platform ID for this environment
  • check if we have some metadata endpoint (or hypervisor back-channel) available for Ignition to use
  • add support in coreos-assembler to buildextend images for this platform
  • add support for ore to upload images
  • add support for afterburn if needed

/cc @imcleod
@ashcrow ashcrow added cloud* related to public/private clouds area/platforms labels Sep 16, 2019
@ashcrow
Copy link
Member Author

ashcrow commented Sep 23, 2019

Keying off #276 (comment)

there doesn't seem to be a provider for "IBM Cloud" in Terraform. How about we use ibmcloud?

@ashcrow
Copy link
Member Author

ashcrow commented Oct 11, 2019

cc @arithx @ajeddeloh @darkmuggle @vrutkovs @lucab

@ashcrow
Copy link
Member Author

ashcrow commented Oct 11, 2019

It looks like ibm is used for the provider via a third party repo.

@ashcrow
Copy link
Member Author

ashcrow commented Oct 11, 2019

In terms of metadata endpoint I've found docs pointing to https://cloudinit.readthedocs.io/en/latest/topics/datasources/configdrive.html (which isn't really an endpoint ..). Still digging.

@arithx
Copy link
Contributor

arithx commented Oct 11, 2019

It looks like ibm is used for the provider via a third party repo.

In that same link they also have a variable called ibmcloud_api_key so I don't think even that is sure what to call it.

I'd personally lean towards ibmcloud

@ashcrow
Copy link
Member Author

ashcrow commented Oct 11, 2019

I'd personally lean towards ibmcloud

I do as well

@lucab
Copy link
Contributor

lucab commented Oct 11, 2019

cloud-init calls it IBMCloud. It seems to be a variant of openstack-configdrive.
https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceIBMCloud.py

@ashcrow
Copy link
Member Author

ashcrow commented Oct 11, 2019

cloud-init calls it IBMCloud. It seems to be a variant of openstack-configdrive.
https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceIBMCloud.py

Side note: That points to config drive as well instead of a metadata endpoint. So far I've been unable to find a metadata endpoint in the docs 🙁.

@ashcrow
Copy link
Member Author

ashcrow commented Oct 15, 2019 •
edited

The other day @darkmuggle found https://cloud.ibm.com/apidocs/vpc. However, I can't seem to fund a location for user-data. I'll continue to look a bit more, but I'm not optimistic that a traditional user-data enabled endpoint is provided.

@tuan-hoang1 do you happen to know if there is a user-data endpoint within IBM Cloud? (a location we can put Ignition data that is retrievable by the instance)

@ashcrow
Copy link
Member Author

ashcrow commented Oct 15, 2019

OK, for the platform id we will go with ibmcloud.

ashcrow added a commit to ashcrow/fedora-coreos-docs that referenced this issue Oct 15, 2019
@ashcrow
pages/platforms: Add ibmcloud to platform list
1cf113d 
See: coreos/fedora-coreos-tracker#277

Signed-off-by: Steve Milner <smilner@redhat.com>
@ashcrow ashcrow mentioned this issue Oct 15, 2019
Merged
@ashcrow
Copy link
Member Author

ashcrow commented Oct 15, 2019

Updating docs: coreos/fedora-coreos-docs#15

@lucab
Copy link
Contributor

lucab commented Oct 21, 2019

The docs/URLs referenced in #277 (comment) are for the Internet-public API, not for link-local metadata.

According to current @ajeddeloh research, we think that there is no metadata endpoint on this cloud provider (which is consistent with the cloud-init implementation I linked above), but we are waiting for confirmation.

@ajeddeloh
Copy link
Contributor

I got confirmation on userdata, though I didn't explicitly ask about other metadata endpoints.

@ashcrow
Copy link
Member Author

ashcrow commented Oct 21, 2019

I got confirmation on userdata, though I didn't explicitly ask about other metadata endpoints.

To clarify, the confirmation is there is no endpoint for userdata and ConfigDrive is the expected solution, correct?

@ajeddeloh
Copy link
Contributor

Correct

ajeddeloh pushed a commit to coreos/fedora-coreos-docs that referenced this issue Oct 23, 2019
@ashcrow
pages/platforms: Add ibmcloud to platform list (#15)
38efadf 
See: coreos/fedora-coreos-tracker#277

Signed-off-by: Steve Milner <smilner@redhat.com>
@lucab
Copy link
Contributor

lucab commented Oct 24, 2019 •
edited

Misc notes from my exploration of the environment so far.

In my IBM cloud console I see there are three types of infrastructure/compute-node available.
Each one has a specific and very different approach to networking (all cases with a floating-ip attached):

  • classic: no DHCP, multiple interfaces, no NAT, legacy interface naming (ethX)
  • VPC gen1: no DHCP, single interface, NAT, legacy interface naming (eth0)
  • VPC gen2: DHCP, single interface, NAT, predictable interface naming (ens3)

None of those of seems to to have reachable link-local metadata endpoint. They all have a local config-drive, however location and content is NOT uniform:

  • classic and gen1: traditional openstack-like VFAT disk with JSON files
  • gen2: iso9660 disk (labeled cidata) with user-data and meta-data files. The latter consists of multiple lines in key-name: value format

Things that I didn't check but would be good followups for the non-DHCP ones:

  • NTP servers
  • DNS servers
  • hostname and domain search-list

@ashcrow
Copy link
Member Author

ashcrow commented Oct 24, 2019

classic and VPC Gen 2 should be supported. It may be worth having different platform id's for these so we can tell them apart easily. Thoughts?

@ajeddeloh
Copy link
Contributor

Having both under one name isn't a problem from the Ignition side. We can do it like the openstack provider and just try both at the same time and see which one succeeds.

@ajeddeloh
Copy link
Contributor

Thinking about this more though, we'd need to more for classic to bring up networking in the initramfs, so having it differentiated might be useful there. cc @lucab @bgilbert

@ashcrow
Copy link
Member Author

ashcrow commented Oct 25, 2019

Thinking about this more though, we'd need to more for classic to bring up networking in the initramfs, so having it differentiated might be useful there. cc @lucab @bgilbert

cc'ing @darkmuggle as well

@darkmuggle
Copy link
Contributor

Breaking the enablement into two separate "clouds" makes sense from a planning and implementation perspective.

@lucab
Copy link
Contributor

lucab commented Nov 1, 2019

After some more chatting and work planning, we decided to start with a reduced scope in order to break a few bootstrapping tasks in an easier way.
For the moment, we are scoping the ibmcloud platform ID to mean "VPC Gen2" only. Reference docs update at coreos/fedora-coreos-docs#16.

Once we get to a reasonable state with "VPC Gen2", we will re-evaluate "classic" infrastructure next and decide whether it is feasible and better (from a technical point of view) to use the same platform ID for both. If not, we'll need to note down the new constraints and start work under a new platform ID.

@lucab lucab mentioned this issue Nov 1, 2019
Merged
This was referenced Feb 10, 2020
Merged
Closed
@lucab
Copy link
Contributor

lucab commented Feb 10, 2020

Reached a further decision point on the chimera-cloud situation, input on coreos/coreos-assembler#1119 appreciated.

@dustymabe dustymabe added the jira for syncing to jira label Jul 27, 2020
@dustymabe dustymabe self-assigned this Aug 22, 2020
@dustymabe
Copy link
Member

dustymabe commented Aug 22, 2020 •
edited

For this I'm going to shoot to an MVP of just getting the images on the website. After that we can worry about auto uploading via ore and CI testing via kola. Here's the list of things to do. I'll update this list as we go:

This list is inspired by the list we had for vultr.

@dustymabe dustymabe mentioned this issue Aug 26, 2020
Open
@dustymabe
Copy link
Member

I broke out the ore/kola work into #614

@dustymabe
Copy link
Member

A few new PRs

Updated #277 (comment) to reflect current status.

@dustymabe
Copy link
Member

A few more new PRs

@dustymabe
Copy link
Member

and a few final ones:

@lucab lucab mentioned this issue Sep 30, 2020
Merged
@dustymabe
Copy link
Member

This is pretty much all done. See updated checklist in #277 (comment)

The only thing left is me pushing the merge button on the docs PR, which we'll do when the round of releases goes out this week.

@dustymabe dustymabe added the status/pending-stable-release Fixed upstream and in testing. Waiting on stable release. label Oct 5, 2020
@dustymabe
Copy link
Member

dustymabe commented Oct 7, 2020 •
edited

The IBM Cloud images first landed in:

  • stable stream release 32.20200923.3.0
  • testing stream release 32.20201004.2.0

@dustymabe dustymabe removed the status/pending-stable-release Fixed upstream and in testing. Waiting on stable release. label Oct 7, 2020
@dustymabe dustymabe mentioned this issue Oct 7, 2020
Closed
@dustymabe dustymabe mentioned this issue Oct 28, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dustymabe dustymabe

Labels
area/platforms cloud* related to public/private clouds jira for syncing to jira
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@ashcrow @lucab @darkmuggle @arithx @ajeddeloh @dustymabe