New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zchunk support #127

Open
wants to merge 5 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@jdieter

jdieter commented Jun 12, 2018

This patchset adds zchunk support to librepo. It requires that libsolv also have zchunk support enabled, and, for it to be effective, the calling application needs to pass the handle a cache directory to find old zchunk files to delta against (I've submitted a patch to dnf).

@jdieter

This comment has been minimized.

jdieter commented Jun 12, 2018

Do please note, that, unlike the libsolv and dnf patches, these patches make zchunk a mandatory dependency. If you'd rather it be an optional dependency, please let me know.

@Conan-Kudo

This comment has been minimized.

Member

Conan-Kudo commented Jun 12, 2018

@jdieter I suspect that it might be desired for the RHEL side of the house to be optional, but I don't see a reason why we wouldn't want it on in Fedora and Mageia.

@j-mracek

This comment has been minimized.

Contributor

j-mracek commented Aug 21, 2018

Without a merge of patch for libsolv it will not work?

@jdieter

This comment has been minimized.

jdieter commented Aug 21, 2018

That's correct. It will download the zchunk metadata, and send it to libsolv, but without openSUSE/libsolv#270, libsolv won't be able to read the zchunk metadata.

@@ -31,6 +31,12 @@ PKG_SEARCH_MODULE(LIBCRYPTO REQUIRED libcrypto openssl)
PKG_CHECK_MODULES(LIBXML2 libxml-2.0 REQUIRED)
FIND_PACKAGE(CURL REQUIRED)
FIND_PACKAGE(Gpgme REQUIRED)
FIND_LIBRARY(ZCHUNKLIB NAMES zck)

This comment has been minimized.

@ignatenkobrain

ignatenkobrain Aug 21, 2018

this should be pkg_check_modules

@dmach

This comment has been minimized.

Contributor

dmach commented Sep 6, 2018

zck.h should #include <stdbool.h> as the boolean type is part of public header

There are also couple compilation errors, such as:
/usr/include/zck.h:62:29: note: expected 'zckCtx *' {aka 'struct zckCtx *'} but argument is of type 'int'
librepo/downloader.c:1089:11: error: too few arguments to function 'zck_init_adv_read
librepo/downloader.c:1156:31: error: too few arguments to function 'zck_init_read'
librepo/downloader.c:1208:29: error: too few arguments to function 'zck_get_range_char'

  • tested with zchunk-devel-0.9.7-2.fc29

I also created PR to make zchunk optional - it's very important to us so we can turn it off on as needed:
jdieter#1

@jdieter

This comment has been minimized.

jdieter commented Sep 18, 2018

Ok, I've gone ahead and merged your PR and added <stdbool.h> to zck.h in zchunk-0.9.10. This now compiles fine with zchunk-0.9.10+, but it won't work with repositories generated using the current createrepo_c zchunk patches because of the format changes.

@jdieter

This comment has been minimized.

jdieter commented Sep 28, 2018

@dmach, If you'd prefer these to be in fewer commits, I'll squash them down for you

@Conan-Kudo

This comment has been minimized.

Member

Conan-Kudo commented Sep 28, 2018

@jdieter Could you please do so?

jdieter and others added some commits May 9, 2018

Add zchunk support
Signed-off-by: Jonathan Dieter <jdieter@gmail.com>
Add zchunk tests
Signed-off-by: Jonathan Dieter <jdieter@gmail.com>
Fix to work with final zchunk API and repomd.xml format
Signed-off-by: Jonathan Dieter <jdieter@gmail.com>
@jdieter

This comment has been minimized.

jdieter commented Oct 4, 2018

@dmach, @Conan-Kudo is there anything else you need me to do on these before review?

@@ -38,6 +43,9 @@ BuildRequires: libcurl-devel >= 7.19.0
BuildRequires: pkgconfig(libxml-2.0)
BuildRequires: pkgconfig(libcrypto)
BuildRequires: pkgconfig(openssl)
%if %{with zchunk}
BuildRequires: zchunk-devel

This comment has been minimized.

@Conan-Kudo

Conan-Kudo Oct 4, 2018

Member

Use pkgconfig(zck) >= 0.9.11 here.

Fix BR and add minimum version for zchunk
Signed-off-by: Jonathan Dieter <jdieter@gmail.com>
@Conan-Kudo

Looks good to me.

@jdieter

This comment has been minimized.

jdieter commented Oct 4, 2018

Thanks!

@Conan-Kudo

This comment has been minimized.

Member

Conan-Kudo commented Oct 9, 2018

@dmach, @j-mracek: This is now merged into libsolv, so can we get this reviewed and merged, as well as the corresponding PRs for the other components of the stack (as well as createrepo_c)?

@jdieter

This comment has been minimized.

jdieter commented Oct 17, 2018

Is there anything else you need from me before you review this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment