CMake incorrect absolute include/lib paths tracking issue #144170

nh2 · 2021-11-02T00:47:51Z

There are various issues where cmake does not work with nix store paths given as prefixes, generating invalid paths into pkg-config .pc files and others.

This issue is to track such cases and their workarounds, so that we can ideally find a general solution, and to figure out whether it's the cmake usage in the upstream packages that's wrong, or something else.

If you find such a case, please post here.

The wrong paths look like this:

NIX_PATH=nixpkgs=. nix-shell -p libyamlcpp -p pkg-config --run 'pkg-config --cflags yaml-cpp'

-I/nix/store/bq106wng1cqk8r4y1y7yh5h7cz49jxpv-libyaml-cpp-0.7.0//nix/store/bq106wng1cqk8r4y1y7yh5h7cz49jxpv-libyaml-cpp-0.7.0/include

Note the concatenation of a store path /nix/store/bq106wng1cqk8r4y1y7yh5h7cz49jxpv-libyaml-cpp-0.7.0/ and the same store path again, and then /include.

Cases

Include/library paths:

hhvm: 029c3f9
srt: srt: Fix wrong srt.pc include path #71669
exiv2: exiv2: fix exiv2.pc file #81091

.cmake paths (see #144170 (comment)):

The text was updated successfully, but these errors were encountered:

Fixes a "double prefix" issue, where parts of the include files for hhvm where located in `$out/$out/include` instead of `$out/include`.

nh2 · 2021-11-02T00:53:50Z

CC from other issues: @andir @j0sh @dtzWill @bennofs @cole-h @jtojnar

nh2 · 2021-11-02T00:54:08Z

Linking from #81091 (comment) the upstream issue:

https://gitlab.kitware.com/cmake/cmake/-/issues/19568

@jtojnar Do you understand the upstream MRs that closed that issue?

r-burns · 2021-11-02T04:35:32Z

The cmake project is more general but has some similar issues: https://github.com/NixOS/nixpkgs/projects/32

r-burns · 2021-11-02T04:45:18Z

The usptream cmake feature that closed that issue is cmake_path (available starting with cmake 3.20)

jtojnar · 2021-11-02T07:05:19Z

The upstream PR basically allows us to simplify the example code in https://github.com/jtojnar/cmake-snips#concatenating-paths-when-building-pkg-config-files to:

cmake_path(APPEND libdir_for_pc_file "\${prefix}" "${CMAKE_INSTALL_LIBDIR}")

configure_file(
  "${PROJECT_SOURCE_DIR}/my-project.pc.in"
  "${PROJECT_BINARY_DIR}/my-project.pc"
  @ONLY)

The downside, as observed above, is that it is only available in CMake 3.20, while projects often have cmake_minimum_required(VERSION 3.x) where x is close to 0. Not sure if they just did not have time to modernize their build systems (modern CMake builds should be based around targets but projects often still use CMake 2 style systems) or they intentionally try to support ancient distros (for example, Debian Jessie whose support ended in June 2020 is stuck on 3.0).

For that reason, using the JoinPaths module from cmake-snips might be still needed for a long time.

The cmake_path also has the disadvantage that the paths are in the format of the build platform (I suspect the docs are using host platform where they mean build platform and target platform where they mean host plaform in GNU terminology):

Note: The cmake_path command handles paths in the format of the build system (i.e. the host platform), not the target system. When cross-compiling, if the path contains elements that are not representable on the host platform (e.g. a drive letter when the host is not Windows), the results will be unpredictable.

So it might accidentally preserve drive letter ( root-name in CMake terminology) when building on Windows for Unix host – for example, when the following project is configured with -DCMAKE_INSTALL_LIBDIR=/usr/lib (and CMAKE_INSTALL_PREFIX defaults to c:/Program Files/${PROJECT_NAME}", ${pkglibdir} will contain c:/usr/lib/my-library.

project(my-library)
include(GNUInstallDirs)
cmake_path(APPEND pkglibdir ${CMAKE_INSTALL_PREFIX} ${CMAKE_INSTALL_LIBDIR} ${PROJECT_NAME})

Or the other way round, on Linux, it will happily append c:/ to an existing path.

Though, to be fair, the function from cmake-snips avoided the first issue by only supporting Unix paths and suffered from the second issue as well.

nh2 · 2021-11-03T23:53:47Z

I also found a similar issue in paths in .cmake files:

#144561 (comment)

I added these to the issue description.

r-burns · 2021-11-04T00:58:57Z

The bpp-* issues are apparently due to them setting their INTERFACE_INCLUDE_DIRECTORIES to $<INSTALL_INTERFACE:$<INSTALL_PREFIX>/${CMAKE_INSTALL_INCLUDEDIR}>, once again appending a not-necessarily-relative directory to the install prefix. This is unnecessary anyway; a relative INSTALL_INTERFACE include dir is already interpreted as relative to the install prefix so $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}> would be more correct.

Artturin · 2022-07-03T15:49:40Z

#181875

r-burns · 2022-07-03T22:32:53Z

#180054

Artturin · 2022-09-08T01:22:53Z

#172347 adds a hook to catch broken .pc files

Fixes NixOS#192617. The breakage is introduced by a glslang update. Now `${glslang}/lib/cmake/OSDependentTargets.cmake` points to `${glslang}/share/glslang/glslang-targets.cmake`. The latter requires the `SPIRV-Tools-opt` target. So the two lines in patch should be moved before `OSDependentTargets.cmake`. This also fixes a breakage introduced by cmake described in NixOS#144170.

Fixes: ``` slang> Broken paths found in a .pc file! /nix/store/23f6pl4nhwl4xx1h35hs64d1kqh9g5sk-slang-1.0g20221020_fdf27a0/share/pkgconfig/sv-lang.pc slang> The following lines have issues (specifically '//' in paths). slang> 2:includedir="${prefix}//nix/store/23f6pl4nhwl4xx1h35hs64d1kqh9g5sk-slang-1.0g20221020_fdf27a0/include" slang> 3:libdir="${prefix}//nix/store/23f6pl4nhwl4xx1h35hs64d1kqh9g5sk-slang-1.0g20221020_fdf27a0/lib" slang> It is very likely that paths are being joined improperly. slang> ex: "${prefix}/@CMAKE_INSTALL_LIBDIR@" should be "@CMAKE_INSTALL_FULL_LIBDIR@" slang> Please see NixOS/nixpkgs#144170 for more details. error: builder for '/nix/store/4w5nq781aw161b9lrk1f59j914ibk368-slang-1.0g20221020_fdf27a0.drv' failed with exit code 1; ```

StillerHarpo · 2022-12-15T15:32:02Z

#206265

bcdarwin · 2022-12-20T21:12:24Z

#207042

258: Fix CMake install dir usage in pkgconfig, honour CMAKE_INSTALL_INCLUDEDIR r=AlanGriffiths a=OPNA2608 1. Current assumption about `CMAKE_INSTALL_<dir>` being relative to `CMAKE_INSTALL_PREFIX` is wrong. ``` Broken paths found in a .pc file! /nix/store/0lfs3w5mgjh5rdlrzwm4h18g2jpmmygx-wlcs-1.4.0/lib/pkgconfig/wlcs.pc The following lines have issues (specifically '//' in paths). 2:exec_prefix=${prefix}//nix/store/0lfs3w5mgjh5rdlrzwm4h18g2jpmmygx-wlcs-1.4.0/bin 3:libexecdir=${prefix}//nix/store/0lfs3w5mgjh5rdlrzwm4h18g2jpmmygx-wlcs-1.4.0/libexec It is very likely that paths are being joined improperly. ex: "${prefix}/`@CMAKE_INSTALL_LIBDIR@"` should be "`@CMAKE_INSTALL_FULL_LIBDIR@"` Please see NixOS/nixpkgs#144170 for more details. ``` `CMAKE_INSTALL_<dir>` must not be assumed to be relative to `CMAKE_INSTALL_PREFIX`, [according to the CMake documentation](https://cmake.org/cmake/help/v3.25/module/GNUInstallDirs.html?highlight=gnuinstalldirs): > It should typically be a path relative to the installation prefix so that it can be converted to an absolute path in a relocatable way (see `CMAKE_INSTALL_FULL_<dir>`). **However, an absolute path is also allowed.** CMake has special `_FULL_` variants of the variables which apply the prefix when needed: > The absolute path generated from the corresponding `CMAKE_INSTALL_<dir>` value. If the value is not already an absolute path, an absolute path is constructed typically by prepending the value of the [`CMAKE_INSTALL_PREFIX`](https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html#variable:CMAKE_INSTALL_PREFIX) variable. --- 2. `CMAKE_INSTALL_INCLUDEDIR` isn't honoured. CMake has `CMAKE_INSTALL_INCLUDEDIR` for specifying where header files shall be installed to. It defaults to `include` (relative to `CMAKE_INSTALL_PREFIX`) so when not specifying an include dir, this is functionally identical to the current hardcoding. Co-authored-by: OPNA2608 <christoph.neidahl@gmail.com>

bcdarwin · 2023-01-12T17:52:33Z

#210399

jeff-hykin · 2023-02-08T20:25:39Z

For others running into this, slapping this into the derivation code worked for my issue:

  installPhase =  ''
    ${pkgs.sd}/bin/sd --string-mode '$${"{prefix}//nix/store"}' '/nix/store' **/*.pc
  '';

It should be pretty robust against false-positives, although it only works when the prefix is literally "prefix"

fix: sha256 didn't match 1.23.1 version (matched 1.8.0) - now matches 1.23.2. fix: 1.23.x failed at install phase NixOS#144170.

2xsaiko · 2023-04-15T00:46:09Z

I encountered one of these when setting up installing CMake config for one of my own projects today. That project is not in nixpkgs but might as well describe it here in case anyone else has to deal with something similar:

It happens for libraries (like this one) that install headers via FILE_SET HEADERS and also install exported targets (the CMake-native mechanism, not pkgconfig). The generated *Targets.cmake file will have a section setting up the file set, which will look something like this, with the wrong paths (should be FILES "${_IMPORT_PREFIX}/include/foo.h"):


if(NOT CMAKE_VERSION VERSION_LESS "3.23.0")
  target_sources(foo
    INTERFACE
      FILE_SET "HEADERS"
      # ...
      FILES "${_IMPORT_PREFIX}//nix/store/.../include/foo.h"
  )
endif()

A workaround I found is to just set CMAKE_INSTALL_INCLUDEDIR to be relative to CMAKE_INSTALL_PREFIX if it's an absolute path. (This might actually be a bug in how CMake generates the file set section of the targets file, since it should probably leave away the ${_IMPORT_PREFIX}/ if the path that comes after is absolute, but I might just as well be doing something wrong that I didn't catch, since I don't know a lot about how this works.)

According to NixOS/nixpkgs#144170, replace STRING(APPEND ...) by CMAKE_PATH(APPEND ...) Remove unnecessary cmakeFlags

This patch fixes Capstone 5 build on NixOS. NixOS's build infrastructure sets CMAKE_INSTALL_{LIB,INCLUDE}DIR to absolute paths. If you append it to ${prefix}, you get the wrong path. NixOS automatically detects it and links this issue: NixOS/nixpkgs#144170

chayleaf · 2023-08-07T21:50:02Z

Found a couple more instances of similar issues just out of the files that weren't garbage collected on my laptop:

/nix/store/hlz9dcyn78bdyf7r67phcqdachgi0h9v-ruby2.7.8-msgpack-1.5.1/nix-support/setup-hook:
addToSearchPath RUBYLIB /nix/store/hlz9dcyn78bdyf7r67phcqdachgi0h9v-ruby2.7.8-msgpack-1.5.1/lib/ruby/gems/2.7.0/gems/msgpack-1.5.1//nix/store/hlz9dcyn78bdyf7r67phcqdachgi0h9v-ruby2.7.8-msgpack-1.5.1/lib/ruby/gems/2.7.0/extensions/x86_64-linux/2.7.0/msgpack-1.5.1

/nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0/share/ocio/setup_ocio.sh:

export DYLD_LIBRARY_PATH="/nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0//nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0/lib:${DYLD_LIBRARY_PATH}"
export LD_LIBRARY_PATH="/nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0//nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0/lib:${LD_LIBRARY_PATH}"
export PATH="/nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0//nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0/bin:${PATH}"
export PYTHONPATH="/nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0//nix/store/wwf32hnxdiagl2zphjbvgwxnqq3h4f0b-opencolorio-2.2.0/lib/python3.10/site-packages:${PYTHONPATH}"

/nix/store/5l495fv1y253z1xqyvg3xf2i5i9psqbs-frei0r-plugins-1.7.0/lib/pkgconfig/frei0r.pc:
libdir=/nix/store/5l495fv1y253z1xqyvg3xf2i5i9psqbs-frei0r-plugins-1.7.0//nix/store/5l495fv1y253z1xqyvg3xf2i5i9psqbs-frei0r-plugins-1.7.0/lib

not sure about this one:

/nix/store/zzab1mabj0xjsq05py3n1lldycwdsxad-rocm-comgr-5.4.4/lib/cmake/amd_comgr/amd_comgr-config.cmake:
include("${AMD_COMGR_PREFIX}//nix/store/zzab1mabj0xjsq05py3n1lldycwdsxad-rocm-comgr-5.4.4/lib/cmake/amd_comgr/amd_comgr-targets.cmake")

/nix/store/vv3vq7fnkbddkln85xm3pl87dbxs9f0x-rocm-opencl-runtime-5.4.4/opencl/include/CL/<header name>:
#include "../../..//nix/store/vv3vq7fnkbddkln85xm3pl87dbxs9f0x-rocm-opencl-runtime-5.4.4/include/CL/<header name>

/nix/store/yavqyq3l299avzqagx64gsf9l1pc2sxy-opencolorio-2.2.0/share/ocio/setup_ocio.sh:
export DYLD_LIBRARY_PATH="/nix/store/yavqyq3l299avzqagx64gsf9l1pc2sxy-opencolorio-2.2.0//nix/store/yavqyq3l299avzqagx64gsf9l1pc2sxy-opencolorio-2.2.0/lib:${DYLD_LIBRARY_PATH}"

from what I've seen, most false positives were either "file://" URLs or cases where the prefix was indeed prepended to an absolute path, but was empty instead of "/nix/store/...". Would it make sense to further increase the checking to all text files? This would be quite a far reaching change

This patch fixes Capstone 5 build on NixOS. NixOS's build infrastructure sets CMAKE_INSTALL_{LIB,INCLUDE}DIR to absolute paths. If you append it to ${prefix}, you get the wrong path. NixOS automatically detects it and links this issue: NixOS/nixpkgs#144170

* Add Python bindings for WASM * Update Python bindings for m68k * Update Python bindings for mos65xx * Update Python bindings for x86 * Add Python bindings for SH * Update CS_* constants in Python bindings * Update constants from ARM auto-sync patch * Fixing TriCore disasm instructions (#2088) * allow absolute CMAKE_INSTALL_*DIR (#2134) This patch fixes Capstone 5 build on NixOS. NixOS's build infrastructure sets CMAKE_INSTALL_{LIB,INCLUDE}DIR to absolute paths. If you append it to ${prefix}, you get the wrong path. NixOS automatically detects it and links this issue: NixOS/nixpkgs#144170 --------- Co-authored-by: Peace-Maker <peace-maker@wcfan.de> Co-authored-by: Bastian Koppelmann <bkoppelmann@users.noreply.github.com> Co-authored-by: chayleaf <chayleaf@protonmail.com>

* Add Python bindings for WASM * Update Python bindings for m68k * Update Python bindings for mos65xx * Update Python bindings for x86 * Add Python bindings for SH * Update CS_* constants in Python bindings * Update constants from ARM auto-sync patch * Fixing TriCore disasm instructions (#2088) * allow absolute CMAKE_INSTALL_*DIR (#2134) This patch fixes Capstone 5 build on NixOS. NixOS's build infrastructure sets CMAKE_INSTALL_{LIB,INCLUDE}DIR to absolute paths. If you append it to ${prefix}, you get the wrong path. NixOS automatically detects it and links this issue: NixOS/nixpkgs#144170 * Disable swift binding const generate * update bindings const * update capstone version * update ChangeLog --------- Co-authored-by: Peace-Maker <peace-maker@wcfan.de> Co-authored-by: Bastian Koppelmann <bkoppelmann@users.noreply.github.com> Co-authored-by: chayleaf <chayleaf@protonmail.com>

nh2 added the 0.kind: bug label Nov 2, 2021

This was referenced Nov 2, 2021

srt: Fix wrong srt.pc include path #71669

Merged

exiv2: fix exiv2.pc file #81091

Merged

nh2 referenced this issue Nov 2, 2021

hhvm: fix location of include files in $out

029c3f9

Fixes a "double prefix" issue, where parts of the include files for hhvm where located in `$out/$out/include` instead of `$out/include`.

nh2 mentioned this issue Nov 2, 2021

libyaml-cpp: Fix wrong paths in pkg-config .pc file. #144169

Merged

12 tasks

jtojnar added this to To do in CMake breakage via automation Nov 2, 2021

r-burns added the 3.skill: sprintable label Nov 4, 2021

kotatsuyaki mentioned this issue Jan 8, 2022

libclc: fix libclc.pc file #153954

Merged

12 tasks

calbrecht mentioned this issue May 2, 2022

intel-graphics-compiler is not being fetched from the cache and the compilation fails #169729

Closed

ck3d mentioned this issue Sep 27, 2022

onnxruntime: fix paths in pkg-config file #193163

Merged

13 tasks

chuangzhu mentioned this issue Oct 19, 2022

ncnn: fix build #196779

Merged

13 tasks

StillerHarpo mentioned this issue Dec 17, 2022

Fix absolute libdirpath builds DCMTK/dcmtk#75

Closed

ryantm mentioned this issue Jan 6, 2023

libversion: 3.0.1 -> 3.0.2 #209269

Merged

13 tasks

tux1c added a commit to tux1c/nixpkgs that referenced this issue Feb 13, 2023

mongoc: 1.23.1 -> 1.23.2

46e828f

fix: sha256 didn't match 1.23.1 version (matched 1.8.0) - now matches 1.23.2. fix: 1.23.x failed at install phase NixOS#144170.

This was referenced Feb 13, 2023

mongoc: 1.23.1 -> 1.23.2 #216210

Closed

mongoc: 1.23.1 -> 1.23.2 #216226

Merged

r-burns mentioned this issue Jun 3, 2023

libclc: fix .pc paths #235823

Merged

12 tasks

Freed-Wu added a commit to lyokha/g3kb-switch that referenced this issue Jul 14, 2023

Update CMakeLists.txt, flake.nix

456069c

According to NixOS/nixpkgs#144170, replace STRING(APPEND ...) by CMAKE_PATH(APPEND ...) Remove unnecessary cmakeFlags

chayleaf mentioned this issue Aug 5, 2023

allow absolute CMAKE_INSTALL_*DIR capstone-engine/capstone#2134

Merged

chayleaf mentioned this issue Aug 6, 2023

cmake: make check-pc-files hook also check .cmake files #247474

Open

12 tasks

hacker1024 mentioned this issue Sep 24, 2023

Use full install paths for pkg-config ompl/ompl#1101

Merged

werner291 mentioned this issue Oct 3, 2023

OMPL on Rolling fails to build lopsided98/nix-ros-overlay#311

Closed

AndersonTorres mentioned this issue Oct 7, 2023

TDE, again! #239953

Draft

12 tasks

vq mentioned this issue Dec 23, 2023

gargoyle: 2019.1.1 -> 2023.1 #276280

Open

13 tasks

bryango mentioned this issue Jan 5, 2024

folly: split outputs to reduce closure sizes w/ cmake fixup #278643

Merged

13 tasks

GaetanLepage mentioned this issue Mar 9, 2024

magic-enum: 0.8.2 -> 0.9.5 #294551

Merged

13 tasks

samueldr added the 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems label Apr 23, 2024

Sigmanificient mentioned this issue May 8, 2024

liboqs: 0.8.0 -> 0.10.0, migrate to by name #310224

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CMake incorrect absolute include/lib paths tracking issue #144170

CMake incorrect absolute include/lib paths tracking issue #144170

nh2 commented Nov 2, 2021 •

edited

nh2 commented Nov 2, 2021 •

edited

nh2 commented Nov 2, 2021

r-burns commented Nov 2, 2021

r-burns commented Nov 2, 2021

jtojnar commented Nov 2, 2021

nh2 commented Nov 3, 2021 •

edited

r-burns commented Nov 4, 2021

Artturin commented Jul 3, 2022 •

edited

r-burns commented Jul 3, 2022

Artturin commented Sep 8, 2022 •

edited

StillerHarpo commented Dec 15, 2022

bcdarwin commented Dec 20, 2022

bcdarwin commented Jan 12, 2023

jeff-hykin commented Feb 8, 2023 •

edited

2xsaiko commented Apr 15, 2023

chayleaf commented Aug 7, 2023 •

edited

CMake incorrect absolute include/lib paths tracking issue #144170

CMake incorrect absolute include/lib paths tracking issue #144170

Comments

nh2 commented Nov 2, 2021 • edited

Cases

nh2 commented Nov 2, 2021 • edited

nh2 commented Nov 2, 2021

r-burns commented Nov 2, 2021

r-burns commented Nov 2, 2021

jtojnar commented Nov 2, 2021

nh2 commented Nov 3, 2021 • edited

r-burns commented Nov 4, 2021

Artturin commented Jul 3, 2022 • edited

r-burns commented Jul 3, 2022

Artturin commented Sep 8, 2022 • edited

StillerHarpo commented Dec 15, 2022

bcdarwin commented Dec 20, 2022

bcdarwin commented Jan 12, 2023

jeff-hykin commented Feb 8, 2023 • edited

2xsaiko commented Apr 15, 2023

chayleaf commented Aug 7, 2023 • edited

nh2 commented Nov 2, 2021 •

edited

nh2 commented Nov 2, 2021 •

edited

nh2 commented Nov 3, 2021 •

edited

Artturin commented Jul 3, 2022 •

edited

Artturin commented Sep 8, 2022 •

edited

jeff-hykin commented Feb 8, 2023 •

edited

chayleaf commented Aug 7, 2023 •

edited