Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support query parameters for disk sas uri #120

Closed
wants to merge 95 commits into from

Conversation

armanamjad
Copy link

No description provided.

p-bo and others added 30 commits May 12, 2022 14:46
Currently translated at 0.5% (5 of 955 strings)

Translation: libguestfs/libguestfs-master
Translate-URL: https://translate.fedoraproject.org/projects/libguestfs/libguestfs-master/cs/
(cherry picked from commit 1886a03)
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: libguestfs/libguestfs-master
Translate-URL: https://translate.fedoraproject.org/projects/libguestfs/libguestfs-master/
(cherry picked from commit 7c7b9f5)
…TURES

On RHEL 7 (rpm-devel-4.11.3-45.el7.x86_64):

rpm-c.c: In function ‘guestfs_int_daemon_rpm_start_iterator’:
rpm-c.c:97:44: error: ‘RPMVSF_MASK_NOSIGNATURES’ undeclared (first use in this function)
   rpmtsSetVSFlags (ts, rpmtsVSFlags (ts) | RPMVSF_MASK_NOSIGNATURES);
                                            ^
rpm-c.c:97:44: note: each undeclared identifier is reported only once for each function it appears in

Fixes: commit aa6f803
(cherry picked from commit bc96e0b)
If the appliance is a QCOW2 image, function get_root_uuid_with_file()
fails to read ext filesystem signature (0x53EF at offset 0x438) from it.
This results in the following error:

libguestfs: error: /usr/lib64/guestfs/appliance/root: appliance is not
an extfs filesystem

The error itself is harmless, but misleading.  So let's skip retrieving
the signature and UUID in case the image contains QCOW2 header.  It's
safe because in this case we'll retrieve it later from RAW image dumped
from that QCOW2 by "qemu-img dd".

Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
(cherry picked from commit ef8c659)
cdrtools writes "CDROM" into the Volume Identifier field in the PVD,
whereas genisoimage and xorriso write "ISOIMAGE".  Recognise either
string as valid in the test.

Fixes: libguestfs#79
Reported-by: David Runge
(cherry picked from commit 0956e8e)
These were added to ocamldep in Jan 2012, over 10 years ago.  They
were not present in RHEL 6, but we don't care about that now.

(cherry picked from virt-v2v commit f6108bbd661d3e922d07b47f00daa901ab846e59)

(cherry picked from commit 0a2d439)
    mltools/tools_utils-c.c: Free keystore after decryption

(cherry picked from commit bf8b876)
Before commit 3a00c4d ("Remove inspection from the C library and
switch to daemon/OCaml implementation") in 2017 the name parameter
passed to add_drive was used by inspection to override the device name
that is determined from fstab.  None of our tools ever actually used
this parameter, and when the inspection code was moved inside the
daemon we stopped using this hint field at all.

So it's no longer used, and likely hasn't been used ever.  Therefore
document that the field does nothing.

Reviewed-by: Laszlo Ersek <lersek@redhat.com>
(cherry picked from commit b4081d0)
Fixes: libguestfs#81
(cherry picked from commit f9babf8)
Currently translated at 2.0% (328 of 16046 strings)

Translation: libguestfs/libguestfs-docs-master
Translate-URL: https://translate.fedoraproject.org/projects/libguestfs/libguestfs-docs-master/es/
(cherry picked from commit 7e9ff9e)
…inux

The documentation currently says that the user should avoid passing
"--selinux-relabel" on the command line if the guest does not support
SELinux. However, the "is_selinux_guest" helper function in
"common/mlcustomize/SELinux_relabel.ml" already turns "--selinux-relabel"
into a no-op if some key SELinux files are absent from the guest, so there
is no need to caution the user.

This change is relevant because the subsequent patches will turn on
"--selinux-relabel" by default, and therefore "is_selinux_guest" will grow
in importance.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1554735
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2075718
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220510102757.14466-2-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 8541db0)
Allow the caller to pass in the option to check for, and to store the
result in a (usually static) variable of their choice.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1794518
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220511122345.14208-2-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 5345d42)
Option "-C" of setfiles(8) causes setfiles(8) to exit with status 1 rather
than status 255 if it encounters relabeling errors, but no other (fatal)
error. Pass "-C" to setfiles(8) in "selinux-relabel", because we don't
want the "selinux-relabel" API to fail if setfiles(8) only encounters
relabeling errors.

(NB even without "-C", setfiles(8) continues traversing the directory
tree(s) and relabeling files across relabeling errors, so this change is
specifically about the exit status.)

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1794518
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220511122345.14208-3-lersek@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit a39b79f)
In https://bugzilla.redhat.com/show_bug.cgi?id=2082806 we've been
tracking an insidious qemu bug which intermittently prevents the
libguestfs appliance from starting.  The symptoms are that SeaBIOS
starts and displays its messages, but the kernel isn't reached.  We
found that the kernel does in fact start, but when it tries to set up
page tables and jump to protected mode it gets a triple fault which
causes the emulated CPU in qemu to reset (qemu exits).

This seems to only affect TCG (not KVM).

Yesterday I found that this is caused by using -cpu max which enables
the "la57" feature (5-level page tables[0]), and that we can make the
problem go away using -cpu max,la57=off.  Note that I still don't
fully understand the qemu bug, so this is only a workaround.

I chose to disable 5-level page tables for both TCG and KVM, partly to
make the patch simpler, and partly because I guess it's not a feature
(ie. 57 bit linear addresses) that is useful for the libguestfs
appliance case, where we have limited physical memory and no need to
run any programs with huge address spaces.

I tested this by running both the direct & libvirt paths overnight.  I
expect that this patch will fail with old qemu/libvirt which doesn't
understand the "la57" feature, but this is only intended as a
temporary workaround.

[0] Article about 5-level page tables as background:
https://lwn.net/Articles/717293/

Thanks: Laszlo Ersek
Fixes: https://answers.launchpad.net/ubuntu/+source/libguestfs/+question/701625
Acked-by: Laszlo Ersek <lersek@redhat.com>
(cherry picked from commit 59d7e6e)
GitHub dropped[1] support for git: protocol, so cloing with "git://"
will fail.  Use "https://" instead.

[1] https://github.blog/2021-09-01-improving-git-protocol-security-github/

Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
(cherry picked from commit 16cf069)
The `git-publish`[1] tool is a wrapper around `git-format-patch` and
`git-send-email`.  It's a handy tool that automates some of the tedious
aspects of manual patch submission:

  - Submitting a patch to the list (with a small config in place) is as
    simple as `git publish`

  - On next revisions, it automatically increments version numbers

  - It auto-copies the list of To: and Cc: from your previous iteration

  - It lets you preview/edit emails before submission

  - You can also use standard `git-format-patch` and `git-send-email`
    options with `git publish`

  - You can send pull requests with `git publish --pull-request`

  - It also provides custom hooks ... and more[2]

[1] https://github.com/stefanha/git-publish
[2] https://github.com/stefanha/git-publish/blob/master/git-publish.pod

Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
(cherry picked from commit 8487e90)
Fixes: commit 53d6c00
(cherry picked from commit feaec24)
Under "REMOTE STORAGE", the "NETWORK BLOCK DEVICE" section already
documents some limitations. Turns out we need to describe a quirky
exception for accessing encrypted RBD disks, too.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2033247
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220518083014.9890-1-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
(cherry picked from commit 544bb0f)
Sometimes generating this file fails.  To help with debugging these
situatons, print the guestfsd.deps file after it has been generated.

(cherry picked from commit bf5fcdb)
Instead of continuing on regardless and failing with a weird error
later, error out early if we don't know about the distro and so cannot
set QUERY_FILES_CMD.  This avoids situations like
libguestfs#81

(cherry picked from commit 4418e63)
The current code for working out the distro uses the ID entry from
/etc/os-release, and then we map those strings into a smaller set of
values (basically, what package manager to use).  However it was
suggested that we should try ID_LIKE first so that distros which act
like other distros would work.  On an Arch Linux 32 system:

ID=arch32
ID_LIKE=arch

See-also: libguestfs#81
Thanks: S D Rausty
(cherry picked from commit 63b722b)
We previously didn't bother to check the return values from any librpm
calls.  In some cases where possibly the RPM database is faulty, this
caused us to return a zero-length list of installed applications (but
no error indication).

One way to reproduce this is given below.  Note this reproducer will
only work when run on a RHEL 8 host (or more specifically, with
rpm <= 4.16):

$ virt-builder fedora-28
$ guestfish -a fedora-28.img -i rm /var/lib/rpm/Packages
$ guestfish --ro -a fedora-28.img -i inspect-list-applications /dev/sda4 -vx
...
chroot: /sysroot: running 'librpm'
error: cannot open Packages index using db5 - Read-only file system (30)
error: cannot open Packages database in
error: cannot open Packages index using db5 - Read-only file system (30)
error: cannot open Packages database in
librpm returned 0 installed packages
...

With this commit we get an error instead:

...
chroot: /sysroot: running 'librpm'
error: cannot open Packages index using db5 - Read-only file system (30)
error: cannot open Packages database in
ocaml_exn: 'internal_list_rpm_applications' raised 'Failure' exception
guestfsd: error: rpmtsInitIterator
guestfsd: => internal_list_rpm_applications (0x1fe) took 0.01 secs
libguestfs: trace: internal_list_rpm_applications = NULL (error)
libguestfs: error: internal_list_rpm_applications: rpmtsInitIterator
libguestfs: trace: inspect_list_applications2 = NULL (error)
libguestfs: trace: inspect_list_applications = NULL (error)
...

Not in this case, but in some cases of corrupt RPM databases it is
possible to recover them by running "rpmdb --rebuilddb" as a guest
command (ie. with guestfs_sh).

See-also: https://bugzilla.redhat.com/show_bug.cgi?id=2089623#c12
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2089623
Fixes: commit c9ee831
Reported-by: Xiaodai Wang
Reported-by: Ming Xie
Acked-by: Laszlo Ersek <lersek@redhat.com>
(cherry picked from commit 488245e)
Found by: make dist && make maintainer-check-extra-dist

Fixes: commit 8487e90
(cherry picked from commit a1b0c41)
CentOS Stream has:

ID_LIKE="rhel fedora"

which confused the existing script.  If there are multiple "likes"
arbitrarily pick the first one in the list.

Fixes: commit 63b722b
(cherry picked from commit 7afbf5e)
In guestfs-tools commit 4fe8a03cd2d3 ('sysprep: remove lvm2's default
"system.devices" file', 2022-04-11), we disabled the use of LVM2's new
"devicesfile" feature, which could interfere with the cloning of virtual
machines.

We suspected in

  https://bugzilla.redhat.com/show_bug.cgi?id=2072493#c6

that the same lvm2 feature could affect the libguestfs appliance itself,
but decided in

  https://bugzilla.redhat.com/show_bug.cgi?id=2072493#c8
  https://bugzilla.redhat.com/show_bug.cgi?id=2072493#c10

that this would not be the case, because "appliance/init" already
constructed a pristine LVM_SYSTEM_DIR.

Unfortunately, that's not enough: due to the "use_devicesfile=1" default
(on RHEL9 anyway), some "lvm" invocation, possibly inside the
lvm-set-filter API, *creates* "$LVM_SYSTEM_DIR/devices/system.devices".
And then we get (minimally) warnings such as

> Please remove the lvm.conf global_filter, it is ignored with the devices
> file.
> Please remove the lvm.conf filter, it is ignored with the devices file.

when using the lvm-set-filter API.

Explicitly disable the "devices file" in "appliance/init", and also
whenever we rewrite "lvm.conf" -- that is, in set_filter()
[daemon/lvm-filter.c]. In the former, check for the feature by locating
the devicesfile-related utilities "lvmdevices" and "vgimportdevices". In
the C code, invoke the utilities with the "--help" option instead. (In
"appliance/init",  I thought it was best not to call any lvm2 utilities
even with "--help", with our lvm2.conf still under construction there.) If
either utility is available, set "use_devicesfile = 0".

Cc: David Teigland <teigland@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1965941
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20220530141027.16167-1-lersek@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
[lersek@redhat.com: style fix: break "devicesfile_feature" in the function
 definition to a new line]
(cherry picked from commit 8fc4d16)
On older GCC:

debug.c:116:32: error: unknown option after ‘#pragma GCC diagnostic’ kind [-Werror=pragmas]
  116 | #pragma GCC diagnostic ignored "-Wanalyzer-mismatching-deallocation"
      |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[3]: *** [Makefile:2039: guestfsd-debug.o] Error 1

The upstream bug (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99193)
has now been fixed so the workaround is not necessary with the latest
GCC, so just delete the workaround.

(cherry picked from commit 1087d31)
@armanamjad armanamjad closed this May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants