Skip to content

Conversation

jovial
Copy link

@jovial jovial commented Apr 28, 2020

markgoddard and others added 30 commits October 25, 2019 13:57
Change-Id: I7820452f107e036d92b282462b0911a0f0e46d82
xfsprogs is required for formatting XFS ephemeral disk partitions
when format=xfs is specified as instance create failes with
'mkfs.xfs: No such file or directory' due to missing xfsprogs
package in the nova-compute container.

Closes-Bug: #1850610

Change-Id: Iaf3414464f3dd747427247339c6b201b352063cb
(cherry picked from commit 9b1a243)
The ICMP probe in the Prometheus blackbox exporter requires
elevated privileges to function. Linux root user or CAP_NET_RAW
capability is required. Can be set by executing setcap
cap_net_raw+ep blackbox_exporter. This change also bumps the version
of the blackbox exporter to the next minor version to allow for this
support.

'+ep' means we're adding the capability as effective and permitted.

See: https://github.com/prometheus/blackbox_exporter

Change-Id: I1c3b817712ae10edd45de01382b044af2fb728bd
Closes-Bug: #1840631
Co-authored-by: Jack Heskett <Jack.Heskett@gresearch.co.uk>
(cherry picked from commit b5a4478)
We broke 'apt_sources_list' and 'apt_preferences' options quite a while
ago. Turned out that fix is quite simple.

This change adds empty sources.list and apt-preferences files which are
always added into images. If kolla-build.conf will use any of APT
related options then those files will be replaced with provided ones.

Change-Id: I7bbdcbd5c60f654a484a872209e0537e3950346c
Closes-bug: #1706549
(cherry picked from commit bf20b41)
This is a combination of 3 commits from Train. Rather than dropping
xtrabackup, we are keeping it in addition to adding support for
backups using mariabackup.

1. Drop Percona XtraBackup in favour of MariaDB Backup

XtraBackup started failing to install on Ubuntu and does not support
MariaDB 10.3 version.

Related-Bug: #1843043

I9cf62c8e03a02574d85e5349df1d23a51d4c99be

2. Fix mariabackup arguments

After switching from innobackupex to mariabackup [1] for database
backups, the kolla_mariadb_backup.sh script was not modified to account
for the difference in arguments between innobackupex and mariabackup.
There is a compatibility mode, the documentation [2] for which covers
some of the differences.

The following have been changed:

- Add explicit --backup argument, now required
- Remove './' positional argument - not required with --stream
- Remove --no-timestamp argument - only supported in innobackupex
  compatibility mode
- Remove --incremental argument - implied by --incremental-history-name
- Remove deprecated --compress argument, which requires qpress to be
  installed (it is not). The stream is now passed through gzip instead
  [3]

[1] https://mariadb.com/kb/en/library/mariabackup-overview/
[2] https://mariadb.com/kb/en/library/mariabackup-options/#-innobackupex
[3] https://mariadb.com/kb/en/library/using-encryption-and-compression-tools-with-mariabackup/

I67cff47cbf56570b8eaeb66092dd87c2769fc8a6
Closes-Bug: #1843043
(cherry picked from commit 65b27d7)

3. Set pipefail in kolla_mariadb_backup.sh

Without this, a failure in mariabackup would be ignored due to the pipe
to gzip.

I84536b302991e8fb04d5fa67ea266bc428437ce1
Related-Bug: #1843043

Change-Id: Idf1aed905a7d9ad461dd6fe17ed199c8d60370e6
manila-share cannot start on debian/ubuntu because python3-cephfs is missing.

Change-Id: Ia592dda4ee20979c701efe6007f236dba9848997
Closes-Bug: #1851692
(cherry picked from commit 1d5a208)
Instead of doing a recursive chown on /srv/node, chown only /srv/node
and any immediate subdirectory under /srv/node.

Change-Id: I8fd93fa77ba9fc63910360300b8783a34bd4e6d5
Closes-Bug: #1851169
(cherry picked from commit 2d122af)
Change-Id: Ia1f17f3bffbd59ad99e46ddd100b88617737f29c
Change-Id: Ib56e62d1fb4d0fc4a6c627b87a929be0bc614d1e
Closes-bug: #1852086
(cherry picked from commit 0186c5e)
In some situations, Keystone bootstrap can fail, and then unhelpfully
fails displaying the error message output by the 'keystone-manage
bootstrap' command. This appears to be due to unprintable control
characters in the error message which prevent the output of the script
from being valid JSON.

This change fixes the issue by piping the output through 'cat -v', which
replaces unprintable characters with control codes.

Change-Id: I82444bc2272311023cc9e92c5a298d1c4c87483b
Closes-Bug: #1855701
(cherry picked from commit bcca3fa)
Affected builds on Ubuntu which had:
npm 3.5.2
while the latest was:
npm 6.13.4
And CentOS had:
npm 5.6.0

This patch runs npm update via npm.
Pinned to 6.x.

Additionally, this moves npm packages prefix (root) to
/usr/local to avoid conflicts with native packages.

Change-Id: Ibaacc1cc478b2b2f8196a5da4eea3570d7d310ff
Closes-bug: #1856699
(cherry picked from commit 308e85a)
Change-Id: I88ebd3db3bf2be0db1c9082b050fe5788d3a7239
Change-Id: I6f6b32e3858f793a4674967e3fa3fcec87d93e6e
Ubuntu/source deployment of several images (horizon, placement-api, zun)
failed with:

+ exec /usr/sbin/apache2 -DFOREGROUND
apache2: Syntax error on line 80 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot

Change-Id: Ie2a1077f7def0743f1403341985e2109aa490026
(cherry picked from commit 932f09b)
When [1] was committed, horizon stopped building for Train.
Analogous patch is proposed to other stable branches.

This patch removes horizon from upper-constraints.

Kolla master is affected directly because we build Train
for CentOS 7 atm.

[1] https://opendev.org/openstack/requirements/commit/3e54878f9c87266e1b9775f72af06de27a3dad65

Co-authored-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I1bd5a40eeef9612d995c81426fe510e89f438725
(cherry picked from commit 8b7ec43)
Change-Id: Ia1ffb7b0834ffd776d2c7ffac40757eafa51ff29
When [1] was committed, neutron stopped building for Train.
Analogous patch is proposed to other stable branches.

This patch removes neutron from upper-constraints.

Kolla master is affected directly because we build Train
for CentOS 7 atm.

1. https://review.opendev.org/#/c/697370/

Change-Id: I944e8e42fef1d359d767cbc6e1c13371ed753f31
As per [1] we should stop compressing the logs sent to swift in order
to get them back readable via a browser.

[1]: http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011875.html

Change-Id: I5b5f323471676317a2898875cbbf297082328fcc
(cherry picked from commit f44b0b7)
Follow up to I5b5f323471676317a2898875cbbf297082328fcc. Logs are no
longer gzipped, so we do not need to modify the 000_FAILED_* symlinks.

Change-Id: I5df42ac0ff91b1b97537a4fbc33fc7970f29d60e
(cherry picked from commit 63f773f)
'rally-manage db' command was deprecated since 0.10.0 [1]. Should use
'rally db' instead.

[1] https://rally.readthedocs.io/en/latest/project_info/release_notes/archive/v0.10.0.html#command-line-interface

Closes-Bug: #1856693

Change-Id: I578f7a6d907d5579180bc64c99c7e6c8edf3afc8
(cherry picked from commit 1114859)
For some reason vmware-nsx releases outside openstack/releases
keeping the neutron version in sync.
Somehow we missed the pin in stein (it is pinned in rocky and train).

Change-Id: Iacd75261d14dc7d9fe89d0db7502831976a6b894
From oslo.config 7.0.0 release notes:

> Positional options are now required by default, to match argparse’s
> default behavior. To revert this behavior (and maintain optional
> positional arguments), you need to explicitly specify positional=True,
> required=False as part of the options definition.

So let's follow.

Change-Id: I77fdbcb905b1a7f2d535c3a37b2899118d1442c5
(cherry picked from commit aaa472b)
Antony Messerli and others added 21 commits March 11, 2020 13:31
Adds ssh clients to Ironic Conductor container for
ansible deploy interface.

Change-Id: Ib995c6ebe5ee76ee1a39e4b5918a79bd61b87a01
Closes-Bug: #1862079
(cherry picked from commit 7234e8b)
When doing a source install, ironic doesn't have python3-systemd as a
dependency and the current ironic-conductor container doesn't install
the required dependencies which causes the ansible deploy interface to
fail with the error, "Systemd bindings do not exist".  This adds the
needed dependencies when doing a source build.  This installs the needed
files for both rpm and deb builds on python 2.x and 3.x.

Change-Id: I5d1bcb0f7d7902b4e122c12697483433bcf52ca2
Closes-Bug: #1861427
(cherry picked from commit 4a67e58)
manual backport from master changes:

- https://review.opendev.org/#/c/712987/
- https://review.opendev.org/#/c/713403

Upstream kibana package contains NodeJS x86-64 binaries so can not be
used directly on non-x86 architectures. I took upstream packages,
removed NodeJS binary from it and added 'nodejs' dependency.

Package is present in my Linaro OBS repository where I keep other
packages needed for aarch64 kolla run. Via APT pinning I mark them as
not wanted so they are not used on any architecture with two exceptions:

1. libvirt - we need fixed version to be able to use ThunderX servers
2. kibana - to be able to run it at all

For x86-64 upstream kibana package is used.

Closes-bug: #1867365

Change-Id: I955619d6992f8714ae753dc155e248d255340ecf
tox will be removed from the base image. Install it before that happens.

This change is made in a simple way that can be easily backported.

Change-Id: I2a24c87aa44c1e03ba76a2fb0bd71caf54eb4211
(cherry picked from commit a81413f)
On AArch64 systems we are unable to deploy Stein due to lack of
mariabackup command.

Change-Id: I00b8c945470a0f63f23f2d0da74eed3189f5d9e5
Closes-bug: #1867750
Change-Id: I9c94d95287ef8956556de5d48b7e88b69457e60f
1. Pin openstacksdk and client libraries in kolla-toolbox

Stein and below only.

openstacksdk released 0.45.0 (and possibly 0.44.0 as well) which dropped
py2 compat but did not advertise the fact. It was amended later but it
will now always affect kolla images until pinned to older.

The observable error is:
"ImportError: No module named queue"

Since Train we pin u-c, but not before that.

This change introduces a version constraint for openstacksdk and client
libs:

- python-novaclient
- python-cinderclient
- python-keystoneclient
- python-glanceclient

2. Make deploy jobs non-voting

For similar reasons to 1., the deploy jobs fail when using the openstack
client for testing. Due to a dependency loop between these two projects,
we must make these jobs non-voting to fix the image build. This should
be reverted ASAP.

3. CI: Add https match for mirror.centos.org to template-overrides

Some CentOS repos (currently one - Ceph NFS Ganesha) use
https://mirror.centos.org instead of http:// and we break
their definition by removing mirrorlist= entry and not inserting baseurl
entry due to a missing match.

Original Change-Id: I7b29f8c6be80820042023d2b85162d5cc9a65f08

Change-Id: I8c2da6437333210a7926f2deb926ab5d1c93c3c6
Closes-Bug: #1870349
The extend_start.sh script for cinder-api uses an incorrect test to
check if the MAX_NUMBER variable is set. This leads to a incorrect
cinder-manage command being run if this variable is not set, making the
database migration fail.

This change fixes the test to properly use the MAX_NUMBER variable.

Change-Id: Ie46da709db0de09a9cf641b3c154275282f213fc
Closes-Bug: #1866827
(cherry picked from commit ff42ff2)
These were temporarily made non-voting in
I8c2da6437333210a7926f2deb926ab5d1c93c3c6 to allow us to fix CI.

Change-Id: I498aa50a0d59d090486d60c2b34b4f2fd077d401
Change-Id: Ie4a85d84aef4dda7acee10c08f1377d521479963
The sensu image became unbuildable, failing to install the ceph plugin:

    Unable to resolve dependencies: sensu-plugins-ceph requires english (= 0.6.3)

This has happened because the author of the 'english' gem has seen fit
to publish a new 0.7.0 release, and 'yank' [1] all previous releases
from rubygems.

This change removes the ceph and process-checks plugins, which both
suffer from a dependency on the yanked english 0.6.3.

[1] https://rubygems.org/gems/english/versions/0.6.3

Change-Id: I9d34413fa10b5a2920aca60b08141bfaddcc7415
Closes-Bug: #1871600
(cherry picked from commit 818dc4d)
Kolla was missing vitrage-persistor image required by Vitrage
for data storage.

Change-Id: Id5e143636f9a81e7294b775f3d8b9134bee58054
Closes-Bug: #1869319
(cherry picked from commit ad06096)
This fixes the issue when keystone-manage output included
backslashes and/or double quotation marks which broke JSON string.

Change-Id: Ifae18c407210c12745d29fc4c95dca69aeafe6a8
Closes-bug: #1866017
(cherry picked from commit 33d184a)
Glance API image lacked nfs programs/utils to be able to mount
NFS exports.
This affected Glance usage with Cinder backend when NFS was used
as Cinder backend.

Closes-Bug: #1868574

Change-Id: I3aac1e6a7f7c5f839b036f15713af949fba206cc
(cherry picked from commit cac9d46)
This is a follow up for [1].

"rally-manage db" command was deprecated since 0.10.0 but Ubuntu ships
0.9.1 for Bionic.

[1] https://bugs.launchpad.net/kolla/+bug/1856693

Closes-Bug: #1872283

Change-Id: I5fb0fc4e6a8bf7ab07abc9c78f0673e3e764b4d1
Change-Id: I9d28bb6f656c39acf36b5b2ee8113b03ea176cd9
@jovial
Copy link
Author

jovial commented Apr 28, 2020

OK, we already had https://review.opendev.org/#/c/692112/ since stackhpc/8.0.1.2. I was confused because the gerrit said the patch was included in 8.0.2, but still good to sync.

@jovial jovial requested a review from Wasaac April 28, 2020 12:15
Copy link

@Wasaac Wasaac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@jovial jovial merged commit ed5ea74 into stackhpc/stein Apr 28, 2020
@jovial jovial deleted the merge-upstream-stein branch April 28, 2020 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.