Oracle: cloud-init openstack local detection too strict for oracle cloud #3215

ubuntu-server-builder · 2023-05-11T15:01:09Z

This bug was originally filed in Launchpad as LP: #1784685

Launchpad details

affected_projects = ['cloud-init (Ubuntu)', 'cloud-init (Ubuntu Xenial)', 'cloud-init (Ubuntu Bionic)']
assignee = chad.smith
assignee_name = Chad Smith
date_closed = 2018-10-02T21:13:35.377862+00:00
date_created = 2018-07-31T17:47:51.962921+00:00
date_fix_committed = 2018-07-31T18:44:18.234309+00:00
date_fix_released = 2018-10-02T21:13:35.377862+00:00
id = 1784685
importance = high
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1784685
milestone = None
owner = chad.smith
owner_name = Chad Smith
private = False
status = fix_released
submitter = chad.smith
submitter_name = Chad Smith
tags = ['regression-proposed', 'verification-done', 'verification-done-bionic', 'verification-done-xenial']
duplicates = []

Launchpad user Chad Smith(chad.smith) wrote on 2018-07-31T17:47:51.962921+00:00

=== Begin SRU Template ===
[Impact]
Cloud-init should detect OpenStack datasource on any OracleCloud instance.
Per the bug, it was falling through the DataSourceNone after upgrade to 18.3.9
and a system reboot.

[Test Case]

Deploy an Oracle cloud bionic instance and validate upgrade/reboot path

cat > setup_proposed.sh <<EOF
#/bin/bash
mirror=http://archive.ubuntu.com/ubuntu
echo deb $mirror $(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
apt-get update -q;
apt-get install -qy cloud-init;
EOF

for vm in '129.146.86.46';
do
   echo '=== BEGIN ' $vm ' ==='
   ssh ubuntu@$vm grep CODENAME /etc/os-release;
   ssh ubuntu@$vm -- dpkg-query --show cloud-init;
   ssh ubuntu@$vm -- cloud-init status --long;
   ssh ubuntu@$vm -- cloud-init --version;
   ssh ubuntu@$vm -- cloud-init analyze show;
   scp setup_proposed.sh ubuntu@$vm:.;
   ssh ubuntu@$vm sudo bash ./setup_proposed.sh 2>&1 | egrep 'cloud-init';
   ssh ubuntu@$vm -- sudo cloud-init clean --logs --reboot;
   echo "After clean reboot, upgrade 18.3.9 should detect OpenStackLocal datasource"
   ssh-keygen -f ".ssh/known_hosts" -R $vm;
   ssh ubuntu@$vm -- cloud-init status --long;
   ssh ubuntu@$vm -- cloud-init --version;
done

[Regression Potential]
This fixes a critical issue in datasource detection on Oracle platforms only.
This should not regress any other platforms and is only a minor code path change
to include Oracle's DMI chassis asset tag as a valid OpenStack datasource type.

[Other Info]
Upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=0df2b42

=== End SRU Template ===

=== Original Description ===

cloud-init 18.3 cannot detect OpenStack datasource on Oracle cloud across reboots.

18.2 properly detects DataSourceOpenstack, but 18.3 added a detect_openstack helper function which fails to detect openstack datasource from the environment or DMI data matching a limited set of conditions:

  - DMI system-product_name in ['OpenStack Nova', 'OpenStack Compute']
      OracleCloud product-name is 'Standard PC (i440FX + PIIX, 1996)'
  - DMI chassis_asset_tag in ['OpenTelekomCloud']
         Oracle's chassis asset tag is 'OracleCloud.com'

- proc/1/environ:product_name == 'OpenStack Nova'
Oracle's /proc/1/environ has no product_name key:
$ sudo cat /proc/1/environ
HOME=/init=/sbin/initrecovery=TERM=linuxdrop_caps=BOOT_IMAGE=/boot/vmlinuz-4.15.0-29-genericPATH=/sbin:/usr/sbin:/bin:/usr/binPWD=/rootmnt=/root

We need a more permissive detect_openstack function to detect Oracle during cloud-init's get_data() method.

ubuntu-server-builder · 2023-05-11T15:01:11Z

Launchpad user Server Team CI bot(server-team-bot) wrote on 2018-07-31T18:44:17.040376+00:00

This bug is fixed with commit 3cee0bf to cloud-init on branch master.
To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=3cee0bf8

ubuntu-server-builder · 2023-05-11T15:01:13Z

Launchpad user Scott Moser(smoser) wrote on 2018-07-31T20:24:46.264044+00:00

fixed in cosmic at 18.3-18-g3cee0bf8-0ubuntu1

ubuntu-server-builder · 2023-05-11T15:01:15Z

Launchpad user Chris Halse Rogers(raof) wrote on 2018-08-01T01:34:05.122681+00:00

Hello Chad, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/18.3-9-g2e62cb8a-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

ubuntu-server-builder · 2023-05-11T15:01:17Z

Launchpad user Chris Halse Rogers(raof) wrote on 2018-08-01T01:55:58.942027+00:00

Hello Chad, or anyone else affected,

Accepted cloud-init into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/18.3-9-g2e62cb8a-0ubuntu1~18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

ubuntu-server-builder · 2023-05-11T15:01:19Z

Launchpad user Chad Smith(chad.smith) wrote on 2018-08-01T19:55:54.075366+00:00

SRU regresion validation logs for Oracle bionic and xenial
Launchpad attachments: lp-1784685.txt

ubuntu-server-builder · 2023-05-11T15:01:21Z

Launchpad user Launchpad Janitor(janitor) wrote on 2018-08-08T01:56:21.499600+00:00

This bug was fixed in the package cloud-init - 18.3-9-g2e62cb8a-0ubuntu1~18.04.2

cloud-init (18.3-9-g2e62cb8a-0ubuntu1~18.04.2) bionic-proposed; urgency=medium

cherry-pick 3cee0bf: oracle: fix detect_openstack to report True on
(LP: #1784685)

cloud-init (18.3-9-g2e62cb8a-0ubuntu1~18.04.1) bionic-proposed; urgency=medium

New upstream snapshot. (LP: #1777912)
- docs: note in rtd about avoiding /tmp when writing files
- ubuntu,centos,debian: get_linux_distro to align with platform.dist
- Fix boothook docs on environment variable name (INSTANCE_I ->
  INSTANCE_ID) [Marc Tamsky]
- update_metadata: a datasource can support network re-config every boot
- tests: drop salt-minion integration test
- Retry on failed import of gpg receive keys.
- tools: Fix run-container when neither source or binary package requested.
- docs: Fix a small spelling error. [Oz N Tiram]
- tox: use simplestreams from git repository rather than bzr.

cloud-init (18.3-0ubuntu1~18.04.1) bionic-proposed; urgency=medium

debian/rules: update version.version_string to contain packaged version.
(LP: #1770712)
debian/patches/openstack-no-network-config.patch
add patch to ignore Openstack network_config from network_data.json by
default
New upstream release. (LP: #1777912)
- release 18.3
- docs: represent sudo:false in docs for user_groups config module
- Explicitly prevent sudo access for user module [Jacob Bednarz]
- lxd: Delete default network and detach device if lxd-init created them.
- openstack: avoid unneeded metadata probe on non-openstack platforms
- stages: fix tracebacks if a module stage is undefined or empty
  [Robert Schweikert]
- Be more safe on string/bytes when writing multipart user-data to disk.
- Fix get_proc_env for pids that have non-utf8 content in environment.
- tests: fix salt_minion integration test on bionic and later
- tests: provide human-readable integration test summary when --verbose
- tests: skip chrony integration tests on lxd running artful or older
- test: add optional --preserve-instance arg to integraiton tests
- netplan: fix mtu if provided by network config for all rendered types
- tests: remove pip install workarounds for pylxd, take upstream fix.
- subp: support combine_capture argument.
- tests: ordered tox dependencies for pylxd install
- util: add get_linux_distro function to replace platform.dist
  [Robert Schweikert]
- pyflakes: fix unused variable references identified by pyflakes 2.0.0.
- - Do not use the systemd_prefix macro, not available in this environment
    [Robert Schweikert]
- doc: Add config info to ec2, openstack and cloudstack datasource docs
- Enable SmartOS network metadata to work with netplan via per-subnet
  routes [Dan McDonald]
- openstack: Allow discovery in init-local using dhclient in a sandbox.
- tests: Avoid using https in httpretty, improve HttPretty test case.
- yaml_load/schema: Add invalid line and column nums to error message
- Azure: Ignore NTFS mount errors when checking ephemeral drive
  [Paul Meyer]
- packages/brpm: Get proper dependencies for cmdline distro.
- packages: Make rpm spec files patch in package version like in debs.
- tools/run-container: replace tools/run-centos with more generic.
- Update version.version_string to contain packaged version.
- cc_mounts: Do not add devices to fstab that are already present.
  [Lars Kellogg-Stedman]
- ds-identify: ensure that we have certain tokens in PATH.
- tests: enable Ubuntu Cosmic in integration tests [Joshua Powers]
- read_file_or_url: move to url_helper, fix bug in its FileResponse.
- cloud_tests: help pylint
- flake8: fix flake8 errors in previous commit.
- typos: Fix spelling mistakes in cc_mounts.py log messages [Stephen Ford]
- tests: restructure SSH and initial connections [Joshua Powers]
- ds-identify: recognize container-other as a container, test SmartOS.
- cloud-config.service: run After snap.seeded.service.
- tests: do not rely on host /proc/cmdline in test_net.py
  [Lars Kellogg-Stedman]
- ds-identify: Remove dupe call to is_ds_enabled, improve debug message.
- SmartOS: fix get_interfaces for nics that do not have addr_assign_type.
- tests: fix package and ca_cert cloud_tests on bionic
- ds-identify: make shellcheck 0.4.6 happy with ds-identify.
- pycodestyle: Fix deprecated string literals, move away from flake8.
- azure: Add reported ready marker file. [Joshua Chan]
- tools: Support adding a release suffix through packages/bddeb.
- FreeBSD: Invoke growfs on ufs filesystems such that it does not prompt.
  [Harm Weites]
- tools: Re-use the orig tarball in packages/bddeb if it is around.
- netinfo: fix netdev_pformat when a nic does not have an address assigned.
- collect-logs: add -v flag, write to stderr, limit journal to single boot.
- IBMCloud: Disable config-drive and nocloud only if IBMCloud is enabled.
- Add reporting events and log_time around early source of blocking time

-- Chad Smith chad.smith@canonical.com Tue, 31 Jul 2018 12:58:16 -0600

ubuntu-server-builder · 2023-05-11T15:01:23Z

Launchpad user Chris Halse Rogers(raof) wrote on 2018-08-08T01:56:39.687829+00:00

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

ubuntu-server-builder · 2023-05-11T15:01:26Z

Launchpad user Launchpad Janitor(janitor) wrote on 2018-08-08T01:57:22.720631+00:00

This bug was fixed in the package cloud-init - 18.3-9-g2e62cb8a-0ubuntu1~16.04.2

cloud-init (18.3-9-g2e62cb8a-0ubuntu1~16.04.2) xenial-proposed; urgency=medium

cherry-pick 3cee0bf: oracle: fix detect_openstack to report True on
(LP: #1784685)

cloud-init (18.3-9-g2e62cb8a-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

New upstream snapshot. (LP: #1777912)
- docs: note in rtd about avoiding /tmp when writing files
- ubuntu,centos,debian: get_linux_distro to align with platform.dist
- Fix boothook docs on environment variable name (INSTANCE_I ->
  INSTANCE_ID) [Marc Tamsky]
- update_metadata: a datasource can support network re-config every boot
- tests: drop salt-minion integration test
- Retry on failed import of gpg receive keys.
- tools: Fix run-container when neither source or binary package requested.
- docs: Fix a small spelling error. [Oz N Tiram]
- tox: use simplestreams from git repository rather than bzr.

cloud-init (18.3-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

debian/rules: update version.version_string to contain packaged version.
(LP: #1770712)
debian/patches/openstack-no-network-config.patch
add patch to ignore Openstack network_config from network_data.json by
default
Refresh patches against upstream:
- azure-use-walinux-agent.patch
- ds-identify-behavior-xenial.patch
New upstream release. (LP: #1777912)
- release 18.3
- docs: represent sudo:false in docs for user_groups config module
- Explicitly prevent sudo access for user module [Jacob Bednarz]
- lxd: Delete default network and detach device if lxd-init created them.
- openstack: avoid unneeded metadata probe on non-openstack platforms
- stages: fix tracebacks if a module stage is undefined or empty
  [Robert Schweikert]
- Be more safe on string/bytes when writing multipart user-data to disk.
- Fix get_proc_env for pids that have non-utf8 content in environment.
- tests: fix salt_minion integration test on bionic and later
- tests: provide human-readable integration test summary when --verbose
- tests: skip chrony integration tests on lxd running artful or older
- test: add optional --preserve-instance arg to integraiton tests
- netplan: fix mtu if provided by network config for all rendered types
- tests: remove pip install workarounds for pylxd, take upstream fix.
- subp: support combine_capture argument.
- tests: ordered tox dependencies for pylxd install
- util: add get_linux_distro function to replace platform.dist
  [Robert Schweikert]
- pyflakes: fix unused variable references identified by pyflakes 2.0.0.
- - Do not use the systemd_prefix macro, not available in this environment
    [Robert Schweikert]
- doc: Add config info to ec2, openstack and cloudstack datasource docs
- Enable SmartOS network metadata to work with netplan via per-subnet
  routes [Dan McDonald]
- openstack: Allow discovery in init-local using dhclient in a sandbox.
- tests: Avoid using https in httpretty, improve HttPretty test case.
- yaml_load/schema: Add invalid line and column nums to error message
- Azure: Ignore NTFS mount errors when checking ephemeral drive
  [Paul Meyer]
- packages/brpm: Get proper dependencies for cmdline distro.
- packages: Make rpm spec files patch in package version like in debs.
- tools/run-container: replace tools/run-centos with more generic.
- Update version.version_string to contain packaged version.
- cc_mounts: Do not add devices to fstab that are already present.
  [Lars Kellogg-Stedman]
- ds-identify: ensure that we have certain tokens in PATH.
- tests: enable Ubuntu Cosmic in integration tests [Joshua Powers]
- read_file_or_url: move to url_helper, fix bug in its FileResponse.
- cloud_tests: help pylint
- flake8: fix flake8 errors in previous commit.
- typos: Fix spelling mistakes in cc_mounts.py log messages [Stephen Ford]
- tests: restructure SSH and initial connections [Joshua Powers]
- ds-identify: recognize container-other as a container, test SmartOS.
- cloud-config.service: run After snap.seeded.service.
- tests: do not rely on host /proc/cmdline in test_net.py
  [Lars Kellogg-Stedman]
- ds-identify: Remove dupe call to is_ds_enabled, improve debug message.
- SmartOS: fix get_interfaces for nics that do not have addr_assign_type.
- tests: fix package and ca_cert cloud_tests on bionic
- ds-identify: make shellcheck 0.4.6 happy with ds-identify.
- pycodestyle: Fix deprecated string literals, move away from flake8.
- azure: Add reported ready marker file. [Joshua Chan]
- tools: Support adding a release suffix through packages/bddeb.
- FreeBSD: Invoke growfs on ufs filesystems such that it does not prompt.
  [Harm Weites]
- tools: Re-use the orig tarball in packages/bddeb if it is around.
- netinfo: fix netdev_pformat when a nic does not have an address assigned.
- collect-logs: add -v flag, write to stderr, limit journal to single boot.
- IBMCloud: Disable config-drive and nocloud only if IBMCloud is enabled.
- Add reporting events and log_time around early source of blocking time
- IBMCloud: recognize provisioning environment during debug boots.
- net: detect unstable network names and trigger a settle if needed
- IBMCloud: improve documentation in datasource.
- sysconfig: dhcp6 subnet type should not imply dhcpv4 [Vitaly Kuznetsov]
- packages/debian/control.in: add missing dependency on iproute2.
- DataSourceSmartOS: add locking of serial device. [Mike Gerdts]
- DataSourceSmartOS: sdc:hostname is ignored [Mike Gerdts]
- DataSourceSmartOS: list() should always return a list [Mike Gerdts]
- schema: in validation, raise ImportError if strict but no jsonschema.
- set_passwords: Add newline to end of sshd config, only restart if
  updated.
- pylint: pay attention to unused variable warnings.
- doc: Add documentation for AliYun datasource. [Junjie Wang]
- Schema: do not warn on duplicate items in commands.
- net: Depend on iproute2's ip instead of net-tools ifconfig or route
- DataSourceSmartOS: fix hang when metadata service is down [Mike Gerdts]
- DataSourceSmartOS: change default fs on ephemeral disk from ext3 to
  ext4. [Mike Gerdts]
- pycodestyle: Fix invalid escape sequences in string literals.
- Implement bash completion script for cloud-init command line
- tools: Fix make-tarball cli tool usage for development
- renderer: support unicode in render_from_file.
- Implement ntp client spec with auto support for distro selection
- Apport: add Brightbox, IBM, LXD, and OpenTelekomCloud to list of clouds.
- tests: fix ec2 integration network metadata validation

-- Chad Smith chad.smith@canonical.com Tue, 31 Jul 2018 13:57:21 -0600

ubuntu-server-builder · 2023-05-11T15:01:28Z

Launchpad user Scott Moser(smoser) wrote on 2018-10-02T21:13:37.418281+00:00

This bug is believed to be fixed in cloud-init in version 18.4. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

ubuntu-server-builder added launchpad Migrated from Launchpad priority Fix soon labels May 11, 2023

ubuntu-server-builder closed this as completed May 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Oracle: cloud-init openstack local detection too strict for oracle cloud #3215

Oracle: cloud-init openstack local detection too strict for oracle cloud #3215

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

Oracle: cloud-init openstack local detection too strict for oracle cloud #3215

Oracle: cloud-init openstack local detection too strict for oracle cloud #3215

Comments

ubuntu-server-builder commented May 11, 2023

Deploy an Oracle cloud bionic instance and validate upgrade/reboot path

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023