Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Completion generation with meson requires valid subgid/subuid #1246

Closed
Foxboron opened this issue Feb 26, 2023 · 11 comments
Closed

Completion generation with meson requires valid subgid/subuid #1246

Foxboron opened this issue Feb 26, 2023 · 11 comments
Labels
1. Bug Something isn't working

Comments

@Foxboron
Copy link
Collaborator

Describe the bug
During building meson will try to create completions and this fails as the user doesn't have a set subuid/subgid in the build environment.

Steps how to reproduce the behaviour
Build toolbox without a subuid/subgid for the user.

Expected behaviour
Building toolbox in a restrictive build environment should work.

Actual behaviour
Building toolbox in a restrictive build environment doesn't work.

Screenshots
If applicable, add screenshots to help explain your problem.

Output of toolbox --version (v0.0.90+)
Latest toolbox release.

@Foxboron Foxboron added the 1. Bug Something isn't working label Feb 26, 2023
@Foxboron
Copy link
Collaborator Author

I think we can do something like this.

λ src main» git diff cmd/completion.go
diff --git a/src/cmd/completion.go b/src/cmd/completion.go
index 26ed378..c71135b 100644
--- a/src/cmd/completion.go
+++ b/src/cmd/completion.go
@@ -32,6 +32,7 @@ var completionCmd = &cobra.Command{
        ValidArgs:             []string{"bash", "fish", "zsh"},
        Args:                  cobra.ExactValidArgs(1),
        RunE:                  completion,
+       PersistentPreRunE:     func(cmd *cobra.Command, args []string) error { return nil },
 }

 func init() {

It would ignore some of the setup done with regard to the configuration and logging, but do we need that for the completion sub command?

@antonc42
Copy link

I'm building toolbox with meson in a Python virtualenv and getting this error. Is it related to this bug? Is there a workaround? I'm not familiar with subuid/subgid.

meson setup -Dprofile_dir=/etc/profile.d builddir
The Meson build system
Version: 1.0.1
Source dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4
Build dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir
Build type: native build
Project name: toolbox
Project version: 0.0.99.4
C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
C linker for the host machine: cc ld.bfd 2.34
Host machine cpu family: x86_64
Host machine cpu: x86_64
Compiler for C supports arguments -print-file-name=libc.so: YES 

meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

A full log can be found at /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-logs/meson-log.txt

Contents of the log file:

Build started at 2023-02-27T08:53:54.119687
Main binary: /home/user/scripts/update.d/venv/bin/python3
Build Options: -Dprofile_dir=/etc/profile.d
Python system: Linux
The Meson build system
Version: 1.0.1
Source dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4
Build dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir
Build type: native build
Project name: toolbox
Project version: 0.0.99.4
-----
Detecting compiler via: cc --version
compiler returned <subprocess.Popen object at 0x7f25c8adc760>
compiler stdout:
cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <subprocess.Popen object at 0x7f25c8a841f0>
linker stdout:
GNU ld (GNU Binutils for Ubuntu) 2.34
Copyright (C) 2020 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 9.4.0
/usr/bin/ld -plugin /usr/lib/gcc/x86_64-linux-gnu/9/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper -plugin-opt=-fresolution=/tmp/ccPCscQX.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/9/../../.. --version -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o

Sanity testing C compiler: cc
Is cross compiler: False.
Sanity check compiler command line: cc sanitycheckc.c -o sanitycheckc.exe -D_FILE_OFFSET_BITS=64
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----
Running test binary command:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/sanitycheckc.exe
C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
C linker for the host machine: cc ld.bfd 2.34
-----
Detecting compiler via: cc --version
compiler returned <subprocess.Popen object at 0x7f25c8adcb80>
compiler stdout:
cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <subprocess.Popen object at 0x7f25c8a463d0>
linker stdout:
GNU ld (GNU Binutils for Ubuntu) 2.34
Copyright (C) 2020 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 9.4.0
/usr/bin/ld -plugin /usr/lib/gcc/x86_64-linux-gnu/9/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper -plugin-opt=-fresolution=/tmp/ccv28i5o.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/9/../../.. --version -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o

Sanity testing C compiler: cc
Is cross compiler: False.
Sanity check compiler command line: cc sanitycheckc.c -o sanitycheckc.exe -D_FILE_OFFSET_BITS=64
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----
Running test binary command:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/sanitycheckc.exe
C compiler for the build machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
C linker for the build machine: cc ld.bfd 2.34
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Running compile:
Working directory:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpfde2cs25
Command line:  cc /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpfde2cs25/testfile.c -o /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpfde2cs25/output.obj -c -D_FILE_OFFSET_BITS=64 -O0 -Wl,--start-group -print-file-name=libc.so -Wl,--end-group 

Code:
 extern int i;
int i;

Compiler stdout:
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc.so

Compiler stderr:
 
Compiler for C supports arguments -print-file-name=libc.so: YES 
Running compile:
Working directory:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpph6_a0qc
Command line:  cc /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpph6_a0qc/testfile.c -E -P -D_FILE_OFFSET_BITS=64 -P -O0 -std=c99 

Code:
 
        #ifdef __has_include
         #if !__has_include("shadow/subid.h")
          #error "Header 'shadow/subid.h' could not be found"
         #endif
        #else
         #include <shadow/subid.h>
        #endif
Compiler stdout:
 
Compiler stderr:
 /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpph6_a0qc/testfile.c:4:12: error: #error "Header 'shadow/subid.h' could not be found"
    4 |           #error "Header 'shadow/subid.h' could not be found"
      |            ^~~~~


meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

@Foxboron
Copy link
Collaborator Author

That error is not relevant for this bug. You are missing headers on your system.

@debarshiray
Copy link
Member

I'm building toolbox with meson in a Python virtualenv and getting this error. Is it related to this bug? Is there a workaround? I'm not familiar with subuid/subgid.

[...]
meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

As @Foxboron said, you are missing some header files and such. Shadow 4.9 introduced a new shared library called libsubid.so and shadow/subid.h is the corresponding header file.

@debarshiray
Copy link
Member

@Foxboron did you notice #1248 ?

I encountered a similar, but not exactly the same, problem when building Toolbx 0.0.99.4 for Fedora.

@Foxboron
Copy link
Collaborator Author

Foxboron commented Mar 1, 2023

@debarshiray I didn't! I'll test it in a day or two :)

@antonc42
Copy link

antonc42 commented Mar 1, 2023

I'm building toolbox with meson in a Python virtualenv and getting this error. Is it related to this bug? Is there a workaround? I'm not familiar with subuid/subgid.
[...]
meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

As @Foxboron said, you are missing some header files and such. Shadow 4.9 introduced a new shared library called libsubid.so and shadow/subid.h is the corresponding header file.

@debarshiray Thanks. I opened a separate bug report: #1247

debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
Having a separate convenience function reduces the indentation levels by
at least one, and sometimes two, and makes it easy to have more detailed
debug logs.

This will make the subsequent commit easier to read.

containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence validating the subordinate user and
group ID ranges.

Unfortunately, some build environments don't have subordinate ID ranges
set up.  Therefore, it's better to avoid validating them when generating
the shell completions, especially, since they are generated by Cobra
itself and subordinate ID ranges are not involved at all.

Note that subordinate ID ranges may be needed when the generated shell
completions are actually used in interactive command line environments.
The shell completions invoke the hidden '__complete' command to get the
results that are presented to the user, and, if needed, the subordinate
ID ranges will continue to be used by podman(1) as part of that.

containers#1246
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence validating the subordinate user and
group ID ranges.

Unfortunately, some build environments don't have subordinate ID ranges
set up.  Therefore, it's better to avoid validating them when generating
the shell completions, especially, since they are generated by Cobra
itself and subordinate ID ranges are not involved at all.

Note that subordinate ID ranges may be needed when the generated shell
completions are actually used in interactive command line environments.
The shell completions invoke the hidden '__complete' command to get the
results that are presented to the user, and, if needed, the subordinate
ID ranges will continue to be used by podman(1) as part of that.

Some changes by Debarshi Ray.

containers#1246
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence validating the subordinate user and
group ID ranges.

Unfortunately, some build environments, like openSUSE's, don't have
subordinate ID ranges set up.  Therefore, it's better to not validate
the subordinate IDs ranges when generating the shell completions, since
they are generated by Cobra itself and subordinate ID ranges are not
involved at all.

Note that subordinate ID ranges may be needed when the generated shell
completions are actually used in interactive command line environments.
The shell completions invoke the hidden '__complete' command to get the
results that are presented to the user, and, if needed, the subordinate
ID ranges will continue to be used by podman(1) as part of that.

Some changes by Debarshi Ray.

containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence validating the subordinate user and
group ID ranges.

Unfortunately, some build environments, like openSUSE's, don't have
subordinate ID ranges set up.  Therefore, it's better to not validate
the subordinate IDs ranges when generating the shell completions, since
they are generated by Cobra itself and subordinate ID ranges are not
involved at all.

Note that subordinate ID ranges may be needed when the generated shell
completions are actually used in interactive command line environments.
The shell completions invoke the hidden '__complete' command to get the
results that are presented to the user, and, if needed, the subordinate
ID ranges will continue to be used by podman(1) as part of that.

Some changes by Debarshi Ray.

containers#1246
containers#1249
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 1, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
@debarshiray
Copy link
Member

@debarshiray I didn't! I'll test it in a day or two :)

The problem that you reported here is different from the one I experienced for Fedora. Turns out that folks at openSUSE are also having the same problem as you.

Anyway, I attempted a fix at #1252 and added a test so that hopefully this doesn't regress.

debarshiray pushed a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence validating the subordinate user and
group ID ranges.

Unfortunately, some build environments, like openSUSE's, don't have
subordinate ID ranges set up.  Therefore, it's better to not validate
the subordinate ID ranges when generating the shell completions, since
they are generated by Cobra itself and subordinate ID ranges are not
involved at all.

Note that subordinate ID ranges may be needed when the generated shell
completions are actually used in interactive command line environments.
The shell completions invoke the hidden '__complete' command to get the
results that are presented to the user, and, if needed, the subordinate
ID ranges will continue to be used by podman(1) as part of that.

Some changes by Debarshi Ray.

containers#1246
containers#1249
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.  eg., the inability to use podman(1) in the
case of Fedora or not having subordinate user and group ID ranges in the
case of openSUSE.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
Having a separate convenience function reduces the indentation levels by
at least one, and sometimes two, and makes it easy to have more detailed
debug logs.

This will make the subsequent commit easier to read.

containers#1246
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
Ever since commit bafbbe8, the shell completions are generated
while building Toolbx using the 'completion' command.  This involves
running toolbox(1) itself, and hence validating the subordinate user and
group ID ranges.

Unfortunately, some build environments, like openSUSE's, don't have
subordinate ID ranges set up.  Therefore, it's better to not validate
the subordinate ID ranges when generating the shell completions, since
they are generated by Cobra itself and subordinate ID ranges are not
involved at all.

Note that subordinate ID ranges may be needed when the generated shell
completions are actually used in interactive command line environments.
The shell completions invoke the hidden '__complete' command to get the
results that are presented to the user, and, if needed, the subordinate
ID ranges will continue to be used by podman(1) as part of that.

Some changes by Debarshi Ray.

containers#1246
containers#1249
debarshiray added a commit to debarshiray/toolbox that referenced this issue Mar 2, 2023
This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1).  These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used.  eg., the inability to use podman(1) in the
case of Fedora or not having subordinate user and group ID ranges in the
case of openSUSE.

It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.

containers/podman#17657
containers#1246
@debarshiray
Copy link
Member

I am closing this because I believe this is fixed by #1252 now. Please re-open if you have reason to believe otherwise. :)

@Foxboron
Copy link
Collaborator Author

Foxboron commented Mar 2, 2023

@debarshiray Everything works after applying all the three relevant commits.

816a7ca
f555029
fc5f568

@debarshiray
Copy link
Member

@debarshiray Everything works after applying all the three relevant commits.

Yay! Thanks for the feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants