-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch from vendoring PCRE2 to downloading and building it #8363
Switch from vendoring PCRE2 to downloading and building it #8363
Conversation
They used to, by accident, but stopped doing that - archlinux/svntogit-community@7e8a186#diff-3e341d2d9c67be01819b25b25d5e53ea3cdf3a38d28846cda85a195eb9b7203a (and the bug report. If a distribution did want to use our pcre2, they would now have to be okay with it being downloaded. Typically distributions don't allow network access on their builders, so they'd probably want a cache of some description. But then, typically distributions don't want vendored dependencies either, so that would be rather weird policy. |
Looks great. I agree that this better go in after the release. Small readme update below. diff --git a/README.rst b/README.rst
index 4d63647d4..cb84aa665 100644
--- a/README.rst
+++ b/README.rst
@@ -147,3 +147,3 @@ Compiling fish requires:
- a curses implementation such as ncurses (headers and libraries)
-- PCRE2 (headers and libraries) - a copy is included with fish
+- PCRE2 (headers and libraries) - optional, this will be downloaded if missing
- gettext (headers and libraries) - optional, for translation support
diff --git a/cmake/PCRE2.cmake b/cmake/PCRE2.cmake
index 281cf573c..2bb9cabc9 100644
--- a/cmake/PCRE2.cmake
+++ b/cmake/PCRE2.cmake
@@ -37,3 +37,6 @@ if(FISH_USE_SYSTEM_PCRE2)
else()
- include(FetchContent)
+ include(FetchContent RESULT_VARIABLE HAVE_FetchContent)
+ if (${HAVE_FetchContent} STREQUAL "NOTFOUND")
+ message(FATAL_ERROR "Please install PCRE2 headers, or CMake >= 3.11 so I can download PCRE")
+ endif()
set(CMAKE_TLS_VERIFY true) output when
|
I've tested this and it seems to work nicely. It should do for the cases where people don't have a pcre2 package installed and want to quickly install a fish. Unless, of course, they don't have internet access when building. I'm not sure that's something we need to support, but if this could support tarballs and you could provide your own that might make it easier (tho in that case you could of course also build pcre2 and install it yourself, but that's system-wide). |
This will break the RHEL/CentOS 7 packages as it stands, because there's no PCRE2 on those platforms (it's in EPEL, but I don't think there's any way of making the fish repositories depend on EPEL). There's various workarounds but I'd prefer to wait until after 3.4.0. |
@zanchey Would it then not simply download pcre2 and build against that? Or is cmake too old?
This is definitely for after 3.4.0. |
The package-building workers don't have internet access, by design. |
Would OpenSUSE be basically happy if this had been done using straight vanilla git submodules? |
The way I have configured the builds at present is that a small script runs on a server I control which basically does everything in the release checklist - builds a tarball, creates a Debian package and an RPM spec, and then uploads them to the right place. This helps ensure that the end-to-end release process is working ok (with a few slight quirks). I'm not sure what would happen with submodules and git clones instead of tarball generation, and I haven't looked into it at all (but I presume it would be possible). |
A real git submodule doesn't seem desirable because it messes with long term we should ask distros to package PCRE2, as it makes little sense to package fish but not PCRE2 |
We're 44% "shell" because it's counting all of pcre2's autocruft!
23f7d03
to
41d3d35
Compare
This switches to using the CMake FetchContent path to dynamically download and build PCRE2, allowing us to drop the vendored sources. The FISH_USE_SYSTEM_PCRE2 CMake option is kept, but if false it now means fetch-and-build PCRE2 rather than building vendored sources. Note FetchContent was introduced in CMake 3.11. That is now a prerequisite for building fish with FISH_USE_SYSTEM_PCRE2 disabled.
Now that PCRE2 is dynamically fetched and built, we can remove the vendored directory. Fixes fish-shell#8355
This ensures we don't link against a system installed libpcre2. Comment in the script why not.
CMake's FetchContent package will check out a git repo and leave permissions as read-only, causing rm to fail. Pass -f so that rm will succeed.
We no longer vendor PCRE2 sources, instead we fetch them from the official repo.
41d3d35
to
3c6581b
Compare
Bravely merging (after applying suggestions from review, thank you all). |
We can make the clone a little lighter-weight and slow down the CMake generation step less with GIT_SHALLOW 1. |
CentOS/RHEL 7 does have PCRE2 packages - they were added sometime after release. I'll add them to the spec file and it should be ok (there's another issue causing breakage on old PCRE2 at present). |
GIT_SHALLOW 1 does lighten the load but for some reason not quite as significantly as I suspected it ought to. Looks like a bug being tracked by CMake. Still, it helps. |
As discussed in #8355 and also in #2985, this removes the vendored PCRE2 sources. The
FISH_USE_SYSTEM_PCRE2
CMake variable (if false) now downloads and build PCRE2 directly, using the FetchContent module.This should probably go in after a 3.4 release.
FetchContent
CMake's FetchContent module works by checking out a git repo at configure time, by default into
_deps/pcre2-src
under the build directory. This directory is 18.5 MB. We then useadd_subdirectory
as normal to pull in PCRE2's CMake build.Advantages
Disadvantages
I don't know how to evaluate the impact on Linux distros that use the vendored PCRE2 like Arch, but I think we want them to stop doing that anyways.