Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enable AliYun datasource by default #2751

Closed
ubuntu-server-builder opened this issue May 10, 2023 · 11 comments
Closed

enable AliYun datasource by default #2751

ubuntu-server-builder opened this issue May 10, 2023 · 11 comments
Labels
launchpad Migrated from Launchpad

Comments

@ubuntu-server-builder
Copy link
Collaborator

This bug was originally filed in Launchpad as LP: #1638931

Launchpad details 
affected_projects = ['cloud-init (Ubuntu)', 'cloud-init (Ubuntu Xenial)', 'cloud-init (Ubuntu Zesty)']
assignee = jingni.wjj
assignee_name = Junjie.Wang
date_closed = 2017-09-23T02:14:19.562521+00:00
date_created = 2016-11-03T14:05:04.694810+00:00
date_fix_committed = 2017-09-23T02:14:19.562521+00:00
date_fix_released = 2017-09-23T02:14:19.562521+00:00
id = 1638931
importance = medium
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1638931
milestone = None
owner = smoser
owner_name = Scott Moser
private = False
status = fix_released
submitter = smoser
submitter_name = Scott Moser
tags = ['verification-done-xenial', 'verification-done-zesty']
duplicates = []

Launchpad user Scott Moser(smoser) wrote on 2016-11-03T14:05:04.694810+00:00

=== Begin SRU Template ===
[Impact]
Ubuntu cloud-images do not identify the Aliyun cloud platform.
Until this change was put in place, they would fail to boot and consume
user-data or metadata there.

With the change in place, an official Ubuntu image from
http://cloud-images.ubuntu.com will work properly on Aliyun.

[Test Case]
The ideal test case is:
a.) download a cloud image, update its cloud-init
The tool 'get-proposed-cloudimg' [1] can be used to do this.

[1] https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/get-proposed-cloudimg

b.) publish that image to Aliyun cloud
c.) boot an instance from that image.
d.) verify function by ssh in verifying /run/cloud-init/result.json shows
that aliyun datasource was used.

Ideally post /var/log/cloud-init.log as an attachment to this bug.

[Regression Potential]
Generally small chance for regression, but a regression would most likely
be seen as an attempt to hit the metadata service at the well known end point
http://100.100.100.200.

This should be fairly well confined as attempts to use the Aliyun datasource
are gated by 'is_aliyun' which does verifies that the dmi product name is
"Alibaba Cloud ECS"

[Other Info]
Upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=4a60af5495

=== End SRU Template ===

As discussed at [1], the addition of AliYun datasource did not enable it by default.
This is because it has no definitive check before polling the network for data.

Lawrence suggested he has a solution for this:
 | in our cloud platform, an instance vm could use an asm
 | instructions cpuid (with eax setted to 0x40000100 ), it will
 | returned our platform id keyword "AliYun" into register ebx and ecx,
 | and if vm is not running on our environment, the value is others. so I
 | guess this is a method to identify our platform ?

--
[1] https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/309614
@ubuntu-server-builder ubuntu-server-builder added the launchpad Migrated from Launchpad label May 10, 2023
@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user lawrence peng(kaihuan-pkh) wrote on 2016-11-24T06:33:51.874710+00:00

hi, Scott, as we discussed before, I need execute a cpuid assembly code in python, so I think I should write a bit of C code which inline the assemly and compile it as a library, then call it in python by using the ctypes lib.

I wonder to know is this plan could be feasible ? because of cloud-init source code is pure python, if I add the C code, it must compile when installed.

the another hand, there are some third library module in python which could execute assembly, maybe I could import these module instead of write by myself ?

thx ~

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Scott Moser(smoser) wrote on 2016-12-13T14:51:14.537421+00:00

I really dont want to pick up a dependency on a library for this. It seems like surely there is some way to get at this information, or possibly we could work together to find a way for the platform to more obviously identify itself.

Other platforms identify themselves in dmi data.
For example, on an openstack system:

$ cat /sys/class/dmi/id/sys_vendor
OpenStack Foundation
$ cat /sys/class/dmi/id/product_name
OpenStack Nova

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Scott Moser(smoser) wrote on 2017-05-23T15:40:22.036893+00:00

The merge proposal attached here
https://code.launchpad.net/~jingni.wjj/cloud-init/+git/cloud-init/+ref/aliyun_dev
shows that AliYun is now identifying themselves in product_name of dmi data.
We no longer need the cpuid/c extension.

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Chris J Arges(arges) wrote on 2017-08-23T12:27:47.637351+00:00

Hello Scott, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-233-ge586fe35-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Chris J Arges(arges) wrote on 2017-08-23T12:31:13.644752+00:00

Hello Scott, or anyone else affected,

Accepted cloud-init into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-233-ge586fe35-0ubuntu1~17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Chad Smith(chad.smith) wrote on 2017-08-29T20:52:30.347223+00:00

Hello Junjie.Wang,

I believe you may be the best equipped to validate that this SRU works without error on AliYun cloud instances.

Can you validate the bug per verification instructions that
cloud-init version 0.7.9-233-ge586fe35-0ubuntu1~16.04.1 works as expected.

Best regards,
Chad

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Chad Smith(chad.smith) wrote on 2017-09-12T21:03:24.862143+00:00

Verifying this bug as it was originally contributed by the AliYun cloud maintainer as 'working' in their environment. Impact is limited only to the AliYun instances in this case and since we don't have access to easily deploy instance in AliYun's cloud, we take Junjie's recommendation here.

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Launchpad Janitor(janitor) wrote on 2017-09-13T01:26:05.837714+00:00

This bug was fixed in the package cloud-init - 0.7.9-233-ge586fe35-0ubuntu1~16.04.1

cloud-init (0.7.9-233-ge586fe35-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  • debian/cloud-init.templates: enable Scaleway cloud.
  • debian/cloud-init.templates: enable Aliyun cloud.
  • drop the following cherry picks, now incorporated in snapshot.
    • debian/patches/cpick-5fb49bac-azure-identify-platform...
    • debian/patches/cpick-003c6678-net-remove-systemd-link...
    • debian/patches/cpick-1cd4323b-azure-remove-accidental...
    • debian/patches/cpick-ebc9ecbc-Azure-Add-network-config...
    • debian/patches/cpick-11121fe4-systemd-make-cloud-final...
  • debian/patches/stable-release-no-jsonschema-dep.patch:
    add patch to remove optional dependency on jsonschema.
  • New upstream snapshot.
    • cloudinit.net: add initialize_network_device function and tests
      [Chad Smith]
    • makefile: fix ci-deps-ubuntu target [Chad Smith]
    • tests: adjust locale integration test to parse default locale.
    • tests: remove 'yakkety' from releases as it is EOL.
    • centos: do not package systemd-fsck drop-in.
    • systemd: make systemd-fsck run after cloud-init.service (LP: #1691489)
    • tests: Add initial tests for EC2 and improve a docstring.
    • locale: Do not re-run locale-gen if provided locale is system default.
    • archlinux: fix set hostname usage of write_file. [Joshua Powers]
    • sysconfig: support subnet type of 'manual'.
    • tools/run-centos: make running with no argument show help.
    • Drop rand_str() usage in DNS redirection detection
      [Bob Aman] (LP: #1088611)
    • sysconfig: use MACADDR on bonds/bridges to configure mac_address
      [Ryan Harper]
    • net: eni route rendering missed ipv6 default route config
      [Ryan Harper] (LP: #1701097)
    • sysconfig: enable mtu set per subnet, including ipv6 mtu
      [Ryan Harper]
    • sysconfig: handle manual type subnets [Ryan Harper]
    • sysconfig: fix ipv6 gateway routes [Ryan Harper]
    • sysconfig: fix rendering of bond, bridge and vlan types.
      [Ryan Harper]
    • Templatize systemd unit files for cross distro deltas. [Ryan Harper]
    • sysconfig: ipv6 and default gateway fixes. [Ryan Harper]
    • net: fix renaming of nics to support mac addresses written in upper
      case. (LP: #1705147)
    • tests: fixes for issues uncovered when moving to python 3.6.
    • sysconfig: include GATEWAY value if set in subnet
      [Ryan Harper]
    • Scaleway: add datasource with user and vendor data for Scaleway.
      [Julien Castets]
    • Support comments in content read by load_shell_content.
    • cloudinitlocal fail to run during boot [Hongjiang Zhang]
    • doc: fix disk setup example table_type options [Sandor Zeestraten]
    • tools: Fix exception handling. [Joonas Kylmälä]
    • tests: fix usage of mock in GCE test.
    • test_gce: Fix invalid mock of platform_reports_gce to return False
      [Chad Smith]
    • test: fix incorrect keyid for apt repository. [Joshua Powers]
    • tests: Update version of pylxd [Joshua Powers]
    • write_files: Remove log from helper function signatures.
      [Andrew Jorgensen]
    • doc: document the cmdline options to NoCloud [Brian Candler]
    • read_dmi_data: always return None when inside a container. (LP: #1701325)
    • requirements.txt: remove trailing white space.
    • Tests: Simplify the check on ssh-import-id [Joshua Powers]
    • tests: update ntp tests after sntp added [Joshua Powers]
    • FreeBSD: Make freebsd a variant, fix unittests and
      tools/build-on-freebsd.
    • FreeBSD: fix test failure
    • FreeBSD: replace ifdown/ifup with "ifconfig down" and "ifconfig up".
      [Hongjiang Zhang]
    • FreeBSD: fix cdrom mounting failure if /mnt/cdrom/secure did not exist.
      [Hongjiang Zhang]
    • main: Don't use templater to format the welcome message
      [Andrew Jorgensen]
    • docs: Automatically generate module docs form schema if present.
      [Chad Smith]
    • debian: fix path comment in /etc/hosts template. [Jens Sandmann]
    • suse: add hostname and fully qualified domain to template.
      [Jens Sandmann]
    • write_file(s): Print permissions as octal, not decimal [Andrew Jorgensen]
    • ci deps: Add --test-distro to read-dependencies to install all deps
      [Chad Smith]
    • tools/run-centos: cleanups and move to using read-dependencies
    • pkg build ci: Add make ci-deps- target to install pkgs
      [Chad Smith]
    • selinux: Allow restorecon to be non-fatal. [Ryan Harper]
    • net: Allow netinfo subprocesses to return 0 or 1 due to selinux.
      [Ryan Harper]
    • net: Allow for NetworkManager configuration [Ryan McCabe]
    • Use distro release version to determine if we use systemd in redhat spec
      [Ryan Harper]
    • net: normalize data in network_state object
    • Integration Testing: tox env, pyxld 2.2.3, and revamp framework
      [Wesley Wiedenmeier]
    • Chef: Update omnibus url to chef.io, minor doc changes. [JJ Asghar]
    • tools: add centos scripts to build and test [Joshua Powers]
    • Drop cheetah python module as it is not needed by trunk [Ryan Harper]
    • rhel/centos spec cleanups.
    • cloud.cfg: move to a template. setup.py changes along the way.
    • Makefile: add deb-src and srpm targets. use PYVER more places.
    • makefile: fix python 2/3 detection in the Makefile [Chad Smith]
    • snap: Removing snapcraft plug line [Joshua Powers]
    • RHEL/CentOS: Fix default routes for IPv4/IPv6 configuration.
      [Andreas Karis]
    • test: Fix pyflakes complaint of unused import. [Joshua Powers]
    • NoCloud: support seed of nocloud from smbios information
      [Vladimir Pouzanov] (LP: #1691772)
    • net: when selecting a network device, use natural sort order
      [Marc-Aurèle Brothier]
    • fix typos and remove whitespace in various docs [Stephan Telling]
    • systemd: Fix typo in comment in cloud-init.target. [Chen-Han Hsiao]
    • Tests: Skip jsonschema related unit tests when dependency is absent.
      [Chad Smith]
    • tools/net-convert.py: support old cloudinit versions by using kwargs.
    • ntp: Add schema definition and passive schema validation.
      [Chad Smith] (LP: #1692916)
    • Fix eni rendering for bridge params that require repeated key for
      values. [Ryan Harper] (LP: #1706752)
    • AliYun: Enable platform identification and enable by default.
      [Junjie Wang] (LP: #1638931)

-- Scott Moser smoser@ubuntu.com Mon, 31 Jul 2017 16:36:16 -0400

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Chris Halse Rogers(raof) wrote on 2017-09-13T01:26:32.553792+00:00

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Launchpad Janitor(janitor) wrote on 2017-09-13T01:27:27.937540+00:00

This bug was fixed in the package cloud-init - 0.7.9-233-ge586fe35-0ubuntu1~17.04.1

cloud-init (0.7.9-233-ge586fe35-0ubuntu1~17.04.1) zesty; urgency=medium

  • debian/cloud-init.templates: enable Scaleway cloud.
  • debian/cloud-init.templates: enable Aliyun cloud.
  • drop the following cherry picks, now incorporated in snapshot.
    • debian/patches/cpick-5fb49bac-azure-identify-platform...
    • debian/patches/cpick-003c6678-net-remove-systemd-link...
    • debian/patches/cpick-1cd4323b-azure-remove-accidental...
    • debian/patches/cpick-ebc9ecbc-Azure-Add-network-config...
    • debian/patches/cpick-11121fe4-systemd-make-cloud-final...
  • debian/patches/stable-release-no-jsonschema-dep.patch:
    add patch to remove optional dependency on jsonschema.
  • New upstream snapshot.
    • cloudinit.net: add initialize_network_device function and tests
      [Chad Smith]
    • makefile: fix ci-deps-ubuntu target [Chad Smith]
    • tests: adjust locale integration test to parse default locale.
    • tests: remove 'yakkety' from releases as it is EOL.
    • centos: do not package systemd-fsck drop-in.
    • systemd: make systemd-fsck run after cloud-init.service (LP: #1691489)
    • tests: Add initial tests for EC2 and improve a docstring.
    • locale: Do not re-run locale-gen if provided locale is system default.
    • archlinux: fix set hostname usage of write_file. [Joshua Powers]
    • sysconfig: support subnet type of 'manual'.
    • tools/run-centos: make running with no argument show help.
    • Drop rand_str() usage in DNS redirection detection
      [Bob Aman] (LP: #1088611)
    • sysconfig: use MACADDR on bonds/bridges to configure mac_address
      [Ryan Harper]
    • net: eni route rendering missed ipv6 default route config
      [Ryan Harper] (LP: #1701097)
    • sysconfig: enable mtu set per subnet, including ipv6 mtu
      [Ryan Harper]
    • sysconfig: handle manual type subnets [Ryan Harper]
    • sysconfig: fix ipv6 gateway routes [Ryan Harper]
    • sysconfig: fix rendering of bond, bridge and vlan types.
      [Ryan Harper]
    • Templatize systemd unit files for cross distro deltas. [Ryan Harper]
    • sysconfig: ipv6 and default gateway fixes. [Ryan Harper]
    • net: fix renaming of nics to support mac addresses written in upper
      case. (LP: #1705147)
    • tests: fixes for issues uncovered when moving to python 3.6.
    • sysconfig: include GATEWAY value if set in subnet
      [Ryan Harper]
    • Scaleway: add datasource with user and vendor data for Scaleway.
      [Julien Castets]
    • Support comments in content read by load_shell_content.
    • cloudinitlocal fail to run during boot [Hongjiang Zhang]
    • doc: fix disk setup example table_type options [Sandor Zeestraten]
    • tools: Fix exception handling. [Joonas Kylmälä]
    • tests: fix usage of mock in GCE test.
    • test_gce: Fix invalid mock of platform_reports_gce to return False
      [Chad Smith]
    • test: fix incorrect keyid for apt repository. [Joshua Powers]
    • tests: Update version of pylxd [Joshua Powers]
    • write_files: Remove log from helper function signatures.
      [Andrew Jorgensen]
    • doc: document the cmdline options to NoCloud [Brian Candler]
    • read_dmi_data: always return None when inside a container. (LP: #1701325)
    • requirements.txt: remove trailing white space.
    • Tests: Simplify the check on ssh-import-id [Joshua Powers]
    • tests: update ntp tests after sntp added [Joshua Powers]
    • FreeBSD: Make freebsd a variant, fix unittests and
      tools/build-on-freebsd.
    • FreeBSD: fix test failure
    • FreeBSD: replace ifdown/ifup with "ifconfig down" and "ifconfig up".
      [Hongjiang Zhang]
    • FreeBSD: fix cdrom mounting failure if /mnt/cdrom/secure did not exist.
      [Hongjiang Zhang]
    • main: Don't use templater to format the welcome message
      [Andrew Jorgensen]
    • docs: Automatically generate module docs form schema if present.
      [Chad Smith]
    • debian: fix path comment in /etc/hosts template. [Jens Sandmann]
    • suse: add hostname and fully qualified domain to template.
      [Jens Sandmann]
    • write_file(s): Print permissions as octal, not decimal [Andrew Jorgensen]
    • ci deps: Add --test-distro to read-dependencies to install all deps
      [Chad Smith]
    • tools/run-centos: cleanups and move to using read-dependencies
    • pkg build ci: Add make ci-deps- target to install pkgs
      [Chad Smith]
    • selinux: Allow restorecon to be non-fatal. [Ryan Harper]
    • net: Allow netinfo subprocesses to return 0 or 1 due to selinux.
      [Ryan Harper]
    • net: Allow for NetworkManager configuration [Ryan McCabe]
    • Use distro release version to determine if we use systemd in redhat spec
      [Ryan Harper]
    • net: normalize data in network_state object
    • Integration Testing: tox env, pyxld 2.2.3, and revamp framework
      [Wesley Wiedenmeier]
    • Chef: Update omnibus url to chef.io, minor doc changes. [JJ Asghar]
    • tools: add centos scripts to build and test [Joshua Powers]
    • Drop cheetah python module as it is not needed by trunk [Ryan Harper]
    • rhel/centos spec cleanups.
    • cloud.cfg: move to a template. setup.py changes along the way.
    • Makefile: add deb-src and srpm targets. use PYVER more places.
    • makefile: fix python 2/3 detection in the Makefile [Chad Smith]
    • snap: Removing snapcraft plug line [Joshua Powers]
    • RHEL/CentOS: Fix default routes for IPv4/IPv6 configuration.
      [Andreas Karis]
    • test: Fix pyflakes complaint of unused import. [Joshua Powers]
    • NoCloud: support seed of nocloud from smbios information
      [Vladimir Pouzanov] (LP: #1691772)
    • net: when selecting a network device, use natural sort order
      [Marc-Aurèle Brothier]
    • fix typos and remove whitespace in various docs [Stephan Telling]
    • systemd: Fix typo in comment in cloud-init.target. [Chen-Han Hsiao]
    • Tests: Skip jsonschema related unit tests when dependency is absent.
      [Chad Smith]
    • tools/net-convert.py: support old cloudinit versions by using kwargs.
    • ntp: Add schema definition and passive schema validation.
      [Chad Smith] (LP: #1692916)
    • Fix eni rendering for bridge params that require repeated key for
      values. [Ryan Harper] (LP: #1706752)
    • AliYun: Enable platform identification and enable by default.
      [Junjie Wang] (LP: #1638931)

-- Scott Moser smoser@ubuntu.com Mon, 31 Jul 2017 16:33:24 -0400

@ubuntu-server-builder
Copy link
Collaborator Author

Launchpad user Scott Moser(smoser) wrote on 2017-09-23T02:14:21.797692+00:00

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
launchpad Migrated from Launchpad
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ubuntu-server-builder