Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(#2013213) (Not only) CI and QoL backports #72

Merged
merged 41 commits into from Mar 29, 2022

Conversation

mrc0mmand
Copy link
Member

@mrc0mmand mrc0mmand commented Mar 8, 2022

So, uh, this turned out to be bigger than I expected, mainly due to the meson-related changes. I could rewrite the patches and thus avoid the need to backport them, but this should make backporting easier in the long run.

mrc0mmand and others added 14 commits March 8, 2022 13:00
Make sure our RPM macros work as intended. Based on the original PR
(#16464) by Mikhail Novosyolov.

Co-authored-by: Mikhail Novosyolov <m.novosyolov@rosalinux.ru>
(cherry picked from commit 55c0951)

Related: #2013213
(cherry picked from commit 8e65d93)

Related: #2013213$
…sector sizes

let's make sure repart works with 4K drives and exotic sector sizes.

(cherry picked from commit 2cd341a)

Related: #2013213
Since util-linux/util-linux@921c7da
libfdisk aligns the last partition (on GPT) for optimal I/O. Let's
account for that.

Fixes: #22606
(cherry picked from commit d490188)

Related: #2013213
so install the respective generator only if we're running with older
lvm versions.

See: https://sourceware.org/git/?p=lvm2.git;a=commit;h=ee8fb0310c53ed003a43b324c99cdfd891dd1a7c
(cherry picked from commit d10d562)

Related: #2013213
Since lvm 2.03.15 the transient units are started without `-r`, thus
disappearing once they finish and breaking the test (which expects them
to remain loaded after finishing). Let's accept `LoadState=not-found` as
a valid result as well to fix this.

Follow-up to: d10d562
See: https://sourceware.org/git/?p=lvm2.git;a=commit;h=fbd8b0cf43dc67f51f86f060dce748f446985855

(cherry picked from commit b034f02)

Related: #2013213
Add dlopen_dw(), dlopen_elf() and dlopen_pcre2() to the dlopen test.
To enable adding dlopen_pcre2(), we move pcre2-dlopen.h/c from
src/journal to src/shared.

(cherry picked from commit ee48779)

Related: #2013213
…output

(cherry picked from commit 0802f62)

Related: #2013213
Allow formatting timestamps as number of seconds since the Epoch for easier
machine parsing.

Fixes: #22567

```
$ systemctl show systemd-journald | grep Timestamp
WatchdogTimestampMonotonic=0
ExecMainStartTimestamp=Sat 2021-12-11 15:25:57 CET
ExecMainStartTimestampMonotonic=13030408
ExecMainExitTimestampMonotonic=0
StateChangeTimestamp=Sat 2021-12-11 15:25:57 CET
StateChangeTimestampMonotonic=13049273
InactiveExitTimestamp=Sat 2021-12-11 15:25:57 CET
InactiveExitTimestampMonotonic=13030430
ActiveEnterTimestamp=Sat 2021-12-11 15:25:57 CET
ActiveEnterTimestampMonotonic=13049273
ActiveExitTimestamp=Sat 2021-12-11 15:25:57 CET
ActiveExitTimestampMonotonic=12997236
InactiveEnterTimestamp=Sat 2021-12-11 15:25:57 CET
InactiveEnterTimestampMonotonic=13028890
ConditionTimestamp=Sat 2021-12-11 15:25:57 CET
ConditionTimestampMonotonic=13029539
AssertTimestamp=Sat 2021-12-11 15:25:57 CET
AssertTimestampMonotonic=13029540

$ systemctl show --timestamp=unix systemd-journald | grep Timestamp
WatchdogTimestampMonotonic=0
ExecMainStartTimestamp=@1639232757
ExecMainStartTimestampMonotonic=13030408
ExecMainExitTimestampMonotonic=0
StateChangeTimestamp=@1639232757
StateChangeTimestampMonotonic=13049273
InactiveExitTimestamp=@1639232757
InactiveExitTimestampMonotonic=13030430
ActiveEnterTimestamp=@1639232757
ActiveEnterTimestampMonotonic=13049273
ActiveExitTimestamp=@1639232757
ActiveExitTimestampMonotonic=12997236
InactiveEnterTimestamp=@1639232757
InactiveEnterTimestampMonotonic=13028890
ConditionTimestamp=@1639232757
ConditionTimestampMonotonic=13029539
AssertTimestamp=@1639232757
AssertTimestampMonotonic=13029540
```

(cherry picked from commit ed4a5b4)

Related: #2013213
(cherry picked from commit b58b4a9)

Related: #2013213
Currently the `--timestamp=` option has no effect on timestamps shown by
`systemctl show`, let's fix that.

Spotted in #22567.

Before:
```
$ systemctl show --timestamp=us+utc systemd-journald | grep Timestamp=
ExecMainStartTimestamp=Sat 2021-12-11 15:25:57 CET
StateChangeTimestamp=Sat 2021-12-11 15:25:57 CET
InactiveExitTimestamp=Sat 2021-12-11 15:25:57 CET
ActiveEnterTimestamp=Sat 2021-12-11 15:25:57 CET
ActiveExitTimestamp=Sat 2021-12-11 15:25:57 CET
InactiveEnterTimestamp=Sat 2021-12-11 15:25:57 CET
ConditionTimestamp=Sat 2021-12-11 15:25:57 CET
AssertTimestamp=Sat 2021-12-11 15:25:57 CET
```

After:
```
$ systemctl show --timestamp=us+utc systemd-journald | grep Timestamp=
ExecMainStartTimestamp=Sat 2021-12-11 14:25:57.177848 UTC
StateChangeTimestamp=Sat 2021-12-11 14:25:57.196714 UTC
InactiveExitTimestamp=Sat 2021-12-11 14:25:57.177871 UTC
ActiveEnterTimestamp=Sat 2021-12-11 14:25:57.196714 UTC
ActiveExitTimestamp=Sat 2021-12-11 14:25:57.144677 UTC
InactiveEnterTimestamp=Sat 2021-12-11 14:25:57.176331 UTC
ConditionTimestamp=Sat 2021-12-11 14:25:57.176980 UTC
AssertTimestamp=Sat 2021-12-11 14:25:57.176980 UTC

```

(cherry picked from commit a59e5c6)

Related: #2013213
It should make it easier to figure out what exactly services do there.
For example, with SYSTEMD_LOG_LEVEL=debug userdbd (v249) prints
```
varlink-5: New incoming message: {"method":"io.systemd.UserDatabase.GetUserRecord","parameters":{}}
```
before it crashes and systemd-resolved prints
```
varlink-21: New incoming message: {"method":"io.systemd.Resolve.ResolveAddress","parameters":{"address":[127,0,0,1],"flags":0,"ifindex":1000000,"family":0}}
```
and those messages are helpful (especially when scripts causing them
aren't clever enough to keep track of random stuff they send to systemd
:-))

(cherry picked from commit bf6ef6b)

Related: #2013213
…ower it to 1G on 32bit systems)

Apparently 2G is too low for various real-life systems. But raising it
universally above 2^32 sounds wrong to me, since that makes no sense on
32bit systems, that we still support.

Hence, let's raise the limit to 32G on 64bit systems, and *lower* it to
1G on 32bit systems.

32G is 4 orders of magnitude higher then the old settings. Let's hope
that's enough for now. Should this not be enough we can raise it
further.

Fixes: #22076
(cherry picked from commit e677041)

Related: #2013213
This queries the sector size from libfdisk instead of assuming 512, and
uses that when converting from bytes to the offset/size values libfdisk
expects.

This is an alternative to Tom Yan's #21823, but prefers using libfdisk's
own ideas of the sector size instead of going directly to the backing
device via ioctls. (libfdisk can after all also operate on regular
files, where the sector size concept doesn't necessarily apply the same
way.)

This also makes the "grain" variable, i.e. how we'll align the
partitions. Previously this was hardcoded to 4K, and that still will be
the minimum grain we use, but should the sector size be larger than that
we'll use the next multiple of the sector size instead.

(cherry picked from commit 994b303)

Related: #2013213
@jamacku
Copy link
Member

jamacku commented Mar 8, 2022

wow

yuwata and others added 15 commits March 8, 2022 14:38
Fixes a regression caused by 3008a6f.

Before the commit, when `mkdir_parents_internal()` is called from `mkdir_p()`,
it uses `_mkdir()` as `flag` is zero. But after the commit, `mkdir_safe_internal()`
is always used. Hence, if the path contains a symlink, it fails with -ENOTDIR.

To fix the issue, this makes `mkdir_p()` calls `mkdir_parents_internal()` with
MKDIR_FOLLOW_SYMLINK flag.

Fixes #22334.

(cherry picked from commit 5117059)

Related: #2013213
The goal is to have the detection of features and paths done first, and
then the build target constructions second.

(cherry picked from commit 65dcf9f)

Related: #2013213
This way we can add the entries more naturally in the same place where
they are defined.

(cherry picked from commit 3f871f1)

Related: #2013213
(cherry picked from commit e4e44a0)

Related: #2013213
(cherry picked from commit 77fcf28)

Related: #2013213
Changing the efi compiler this way doesn't really work. The gnu-efi
header checks as well as supported compiler flag checks use the
regular cc that meson detects. Changing the compiler this way will
end up with bad compiler flags. For the very same reason, this does
not work with a cross-compiler without going through proper meson
cross-compilation steps either.

The proper way to build systemd-boot with a different compiler is to
use a different build folder and then just use the proper ninja build
target to only build the bootloader/stub.

(cherry picked from commit 52adf0e)

Related: #2013213
(cherry picked from commit 2f2b072)

Related: #2013213
This fixes build reproducibility as otherwise the full path
of the source files ends up in the output binary.

(cherry picked from commit b3c5a70)

Related: #2013213
Not having to provide the full path in the source tree is much
nicer and the produced lists can also be used anywhere in the source
tree.

(cherry picked from commit e1eeebb)

Related: #2013213
(cherry picked from commit 4b65fc8)

Related: #2013213
Not having to provide the full path in the source tree is much
nicer and the produced lists can also be used anywhere in the source
tree.

(cherry picked from commit bbec46c)

Related: #2013213
(cherry picked from commit 68a06b3)

Related: #2013213
No need to invoke ls when we are just interested in file names.
Also, the cd to source root makes the output identical to
"git ls-files" (relative instead of absolute paths).

(cherry picked from commit 3a46980)

Related: #2013213
(cherry picked from commit 6f6b017)

Related: #2013213
mrc0mmand and others added 6 commits March 8, 2022 14:38
Since the TEST-64-UDEV-STORAGE fails are quite frequent now and the root
cause is yet to be discovered, let's add a kludge that attempts to retry
the test up to two more times in case it fails, so we don't
unnecessarily disturb CIs while the issue is being investigated.

Revert this commit once #21819 is sorted out.

(cherry picked from commit 95e3551)

Related: #2013213
(cherry picked from commit 888d0bc)

Related: #2013213
Avoid doing stack allocations on environment variables.

(cherry picked from commit 1d3b68f)

Related: #2013213
Remove incorrect claim that C escapes (such as \t and \n) are recognized and that control characters are disallowed. Specify the allowed characters and escapes with single quotes, with double quotes, and without quotes.

(cherry picked from commit 4bbcde8)

Related: #2013213
Also rename variables.

(cherry picked from commit 398a500)

Related: #2013213
@mrc0mmand mrc0mmand force-pushed the ci-backports-main branch 2 times, most recently from d9f89ff to a9a82b9 Compare March 8, 2022 17:35
@systemd-rhel-bot systemd-rhel-bot added the pr/needs-review Formerly needs-review label Mar 9, 2022
@systemd-rhel-bot systemd-rhel-bot changed the title (Not only) CI and QoL backports (#2013213) (Not only) CI and QoL backports Mar 9, 2022
(cherry picked from commit e9e982a)

Related: #2013213
@systemd-rhel-bot systemd-rhel-bot added pr/needs-ci Formerly needs-ci and removed pr/needs-ci Formerly needs-ci labels Mar 9, 2022
@jamacku jamacku added this to the RHEL-9.1.0 milestone Mar 25, 2022
Copy link
Member

@dtardon dtardon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@jamacku jamacku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrc0mmand - referenced bug #2013213 is for RHEL 9.0.0 and PR is targeted at main, is it correct?

@systemd-rhel-bot systemd-rhel-bot removed the pr/needs-review Formerly needs-review label Mar 29, 2022
@systemd-rhel-bot systemd-rhel-bot merged commit b4f404c into redhat-plumbers:main Mar 29, 2022
@mrc0mmand
Copy link
Member Author

@mrc0mmand - referenced bug #2013213 is for RHEL 9.0.0 and PR is targeted at main, is it correct?

Yup. The original goal was to have one CI-related tracker for each major RHEL release (at least that's what @dtardon envisioned). Hopefully it still holds.

@jamacku
Copy link
Member

jamacku commented Mar 29, 2022

Yup. The original goal was to have one CI-related tracker for each major RHEL release (at least that's what @dtardon envisioned). Hopefully it still holds.

@mrc0mmand But I'm not sure if we would be able to pass checks on C9S MR.

@dtardon
Copy link
Member

dtardon commented Mar 30, 2022

@mrc0mmand - referenced bug #2013213 is for RHEL 9.0.0 and PR is targeted at main, is it correct?

Yup. The original goal was to have one CI-related tracker for each major RHEL release (at least that's what @dtardon envisioned). Hopefully it still holds.

It does, but the problem here is that the referenced bug is not the RHEL-9 CI tracker... (Yeah, it's partly my fault too, as I should have checked it.)

@mrc0mmand
Copy link
Member Author

@mrc0mmand - referenced bug #2013213 is for RHEL 9.0.0 and PR is targeted at main, is it correct?

Yup. The original goal was to have one CI-related tracker for each major RHEL release (at least that's what @dtardon envisioned). Hopefully it still holds.

It does, but the problem here is that the referenced bug is not the RHEL-9 CI tracker... (Yeah, it's partly my fault too, as I should have checked it.)

🤦 I have no idea where I came up with that bug number then, sigh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet