Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instantiating a ROOT.vector adds the build directory to cling's include paths #7108

Closed
1 task done
eguiraud opened this issue Jan 28, 2021 · 36 comments · Fixed by #9520 or #9533
Closed
1 task done

Instantiating a ROOT.vector adds the build directory to cling's include paths #7108

eguiraud opened this issue Jan 28, 2021 · 36 comments · Fixed by #9520 or #9533

Comments

@eguiraud
Copy link
Member

  • Checked for duplicates

Describe the bug

A reproducer:

import ROOT
print(ROOT.gSystem.GetIncludePath())
ROOT.vector('int')()
print(ROOT.gSystem.GetIncludePath())

prints

-I$ROOTSYS/include -I"/home/eguiraud/ROOT/install_dbg_includepaths/etc/" -I"/home/eguiraud/ROOT/install_dbg_includepaths/etc//cling" -I"/home/eguiraud/ROOT/install_dbg_includepaths/include/" -I"/usr/include/python2.7"
-I$ROOTSYS/include -I"/home/eguiraud/ROOT/install_dbg_includepaths/etc/" -I"/home/eguiraud/ROOT/install_dbg_includepaths/etc//cling" -I"/home/eguiraud/ROOT/install_dbg_includepaths/include/" -I"/usr/include/python2.7" -I"/home/eguiraud/ROOT/build_dbg_includepaths/include"

Note that the second line contains -I"/home/eguiraud/ROOT/build_dbg_includepaths/include", which wasn't there before a ROOT.vector was instantiated. That's problematic because the user might not have permissions to access the build directory (while it can access the install directory) leading to cling errors.

This is with ROOT master@028fcca0fa , compiling ROOT with cmake -DCMAKE_BUILD_TYPE=Debug -Dccache=ON -Ddev=ON -Droofit=OFF -Dtmva=OFF -Dtesting=ON -Droottest=ON ../root.

@vepadulano
Copy link
Member

vepadulano commented Jan 29, 2021

I can't reproduce on my laptop (Fedora 32) with the following 4 config:

  • Python 2.7.18 , Python 3.8.7
  • ROOT master@028fcca compiled with cmake -GNinja -Ddev=ON -DCMAKE_BUILD_TYPE=Debug -Dtesting=ON -Droottest=ON -DCMAKE_INSTALL_PREFIX=$ROOT_INSTALL/${PWD##*/} $ROOT_HOME or with cmake -GNinja -Ddistcc=ON -Ddev=ON -Droot7=OFF -Droofit=OFF -Dtmva=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$ROOT_INSTALL/${PWD##*/} $ROOT_HOME

Python2 and ROOT release

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 --version
Python 2.7.18
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python2.7"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python2.7"

Python2 and ROOT debug

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 --version
Python 2.7.18
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python2 reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python2.7"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python2.7"

Python3 and ROOT release

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python --version
Python 3.8.7
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python3.8"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devrelease/include/" -I"/usr/include/python3.8"

Python3 and ROOT debug

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python --version
Python 3.8.7
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python3.8"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/devdebugtest/include/" -I"/usr/include/python3.8"

@etejedor
Copy link
Contributor

I can reproduce with Enrico's reproducer, but not with:

import ROOT
print(ROOT.gSystem.GetIncludePath())

ROOT.gInterpreter.ProcessLine("""
auto v = std::vector<int>();
""")
print(ROOT.gSystem.GetIncludePath())

So it could be related to the use of the TCling interfaces.

@etejedor
Copy link
Contributor

Instantiating the template is enough to reproduce (no need to create the object):

import ROOT
print(ROOT.gSystem.GetIncludePath())
ROOT.vector('int')
print(ROOT.gSystem.GetIncludePath())

@etejedor
Copy link
Contributor

Using cppyy only has the extra path both before and after:

import cppyy
print(cppyy.gbl.gSystem.GetIncludePath())
cppyy.gbl.std.vector('int')
print(cppyy.gbl.gSystem.GetIncludePath())

@eguiraud
Copy link
Member Author

I guess that means that the path is added by import cppyy. Does import ROOT not imply import cppyy?

@etejedor
Copy link
Contributor

Yes, import ROOT imports cppyy too, so I don't think it's the cppyy import. And if you add import ROOT at the beginning of my last reproducer the first time we get the include paths is correct.

@eguiraud
Copy link
Member Author

@vepadulano can you try with makefiles instead of ninja? I don't see any other difference

@etejedor
Copy link
Contributor

More info: it's not a vector thing, it also happens if you instantiate e.g. a map or list or deque.

@etejedor
Copy link
Contributor

This line in ROOT/__init__.py prevents cppyy from adding the path from the beginning.

`environ['CPPYY_API_PATH'] = 'none'`

but still does not explain why/where the instantiation adds the path.

@eguiraud
Copy link
Member Author

the wrong include path is added here:

#0  0x00007fffefcba72e in cling::Interpreter::AddIncludePaths (this=0x5555559ac2f0, PathStr=..., Delm=0x0) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Interpreter/Interpreter.cpp:626
#1  0x00007fffefcba926 in cling::Interpreter::AddIncludePath (this=0x5555559ac2f0, PathsStr=...) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Interpreter/Interpreter.cpp:652
#2  0x00007fffefacc76d in TCling::AddIncludePath (this=0x555555958c90, path=0x7fffe5517ef8 "/home/eguiraud/ROOT/build_dbg_includepaths/include") at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:2644
#3  0x00007fffefac988e in TCling::RegisterModule (this=0x555555958c90, modulename=0x7fffe5517fce "libvectorDict", headers=0x7fffe552f190 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::headers>, includePaths=0x7fffe552f1a0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::includePaths>, payloadCode=0x7fffe5517f30 "\n#line 1 \"libvectorDict dictionary payload\"\n\n\n#define _BACKWARD_BACKWARD_WARNING_H\n// Inline headers\n#include \"vector\"\n\n#undef  _BACKWARD_BACKWARD_WARNING_H\n", fwdDeclsCode=0x7fffe5517f2b "", triggerFunc=0x7fffe54fff24 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()>, fwdDeclsArgToSkip=std::vector of length 0, capacity 0, classesHeaders=0x7fffe55301d0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::classesHeaders>, lateRegistration=false, hasCxxModule=false) at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:2061
#4  0x00007ffff6f8797c in TROOT::RegisterModule (modulename=0x7fffe5517fce "libvectorDict", headers=0x7fffe552f190 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::headers>, includePaths=0x7fffe552f1a0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::includePaths>, payloadCode=0x7fffe5517f30 "\n#line 1 \"libvectorDict dictionary payload\"\n\n\n#define _BACKWARD_BACKWARD_WARNING_H\n// Inline headers\n#include \"vector\"\n\n#undef  _BACKWARD_BACKWARD_WARNING_H\n", fwdDeclCode=0x7fffe5517f2b "", triggerFunc=0x7fffe54fff24 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()>, fwdDeclsArgToSkip=std::vector of length 0, capacity 0, classesHeaders=0x7fffe55301d0 <(anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl()::classesHeaders>, hasCxxModule=false) at ../../root_dbg_includepaths/core/base/src/TROOT.cxx:2561
#5  0x00007fffe54fff93 in (anonymous namespace)::TriggerDictionaryInitialization_libvectorDict_Impl () at G__vectorDict.cxx:982
#6  0x00007fffe54fffdf in (anonymous namespace)::DictInit::DictInit (this=0x7fffe55301d9 <(anonymous namespace)::__TheDictionaryInitializer>) at G__vectorDict.cxx:990
#7  0x00007fffe550011e in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at G__vectorDict.cxx:992
#8  0x00007fffe5500134 in _GLOBAL__sub_I_G__vectorDict.cxx(void) () at G__vectorDict.cxx:996
#9  0x00007ffff7fe437a in call_init (l=<optimized out>, argc=argc@entry=2, argv=argv@entry=0x7fffffffe4a8, env=env@entry=0x555556b4eb50) at dl-init.c:72
#10 0x00007ffff7fe4476 in call_init (env=0x555556b4eb50, argv=0x7fffffffe4a8, argc=2, l=<optimized out>) at dl-init.c:30
#11 _dl_init (main_map=0x555558799bf0, argc=2, argv=0x7fffffffe4a8, env=0x555556b4eb50) at dl-init.c:119
#12 0x00007ffff7fe82d3 in dl_open_worker (a=a@entry=0x7fffffffc6c0) at dl-open.c:517
#13 0x00007ffff7b4fb2f in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:196
#14 0x00007ffff7fe7bba in _dl_open (file=0x555558d0abf0 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", mode=-2147483391, caller_dlopen=0x7fffefed6582 <cling::utils::platform::DLOpen(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)+41>, nsid=<optimized out>, argc=2, argv=0x7fffffffe4a8, env=0x555556b4eb50) at dl-open.c:599
#15 0x00007ffff7f83256 in dlopen_doit (a=a@entry=0x7fffffffc8e0) at dlopen.c:66
#16 0x00007ffff7b4fb2f in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffc880, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:196
#17 0x00007ffff7b4fbbf in __GI__dl_catch_error (objname=0x5555559ab150, errstring=0x5555559ab158, mallocedp=0x5555559ab148, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:215
#18 0x00007ffff7f83975 in _dlerror_run (operate=operate@entry=0x7ffff7f83200 <dlopen_doit>, args=args@entry=0x7fffffffc8e0) at dlerror.c:163
#19 0x00007ffff7f832e6 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#20 0x00007fffefed6582 in cling::utils::platform::DLOpen (Path="/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", Err=0x7fffffffc990) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Utils/PlatformPosix.cpp:118
#21 0x00007fffefca7d0d in cling::DynamicLibraryManager::loadLibrary (this=0x555555a6e340, libStem="/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", permanent=true, resolved=false) at ../../../../../../../../root_dbg_includepaths/interpreter/cling/lib/Interpreter/DynamicLibraryManager.cpp:184
#22 0x00007fffefacfb91 in TCling::Load (this=0x555555958c90, filename=0x555558d06590 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", system=true) at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:3448
#23 0x00007ffff700d7c1 in TSystem::Load (this=0x55555597b4d0, module=0x555558d064f0 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", entry=0x0, system=true) at ../../../root_dbg_includepaths/core/base/src/TSystem.cxx:1940
#24 0x00007ffff711f8a1 in TUnixSystem::Load (this=0x55555597b4d0, module=0x555558d064f0 "/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so", entry=0x0, system=true) at ../../../root_dbg_includepaths/core/unix/src/TUnixSystem.cxx:2789
#25 0x00007ffff6f86764 in TROOT::LoadClass (this=0x7ffff73a6bc0 <ROOT::Internal::GetROOT1()::alloc>, libname=0x555558cea6e0 "libvectorDict.so", check=false) at ../../root_dbg_includepaths/core/base/src/TROOT.cxx:2148
#26 0x00007fffefada682 in TCling::ShallowAutoLoadImpl (cls=0x555558cf04e0 "vector<bool>") at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:6052
#27 0x00007fffefada7c2 in TCling::DeepAutoLoadImpl (cls=0x555558cf04e0 "vector<bool>") at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:6075
#28 0x00007fffefadae02 in TCling::AutoLoad (this=0x555555958c90, cls=0x555558cf04e0 "vector<bool>", knowDictNotLoaded=true) at ../../../../root_dbg_includepaths/core/metacling/src/TCling.cxx:6191
#29 0x00007ffff70b1520 in TClass::LoadClassDefault (requestedname=0x555558cf04e0 "vector<bool>") at ../../../root_dbg_includepaths/core/meta/src/TClass.cxx:5763
#30 0x00007ffff70a7f0e in TClass::GetClass (name=0x555558cf3170 "std::vector<bool>", load=true, silent=true, hint_pair_offset=0, hint_pair_size=0) at ../../../root_dbg_includepaths/core/meta/src/TClass.cxx:3079
#31 0x00007ffff70a799d in TClass::GetClass (name=0x555558cf3170 "std::vector<bool>", load=true, silent=true) at ../../../root_dbg_includepaths/core/meta/src/TClass.cxx:2948
#32 0x00007ffff740a172 in Cppyy::GetScope (sname="std::vector<bool>") at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:534
#33 0x00007ffff75602cb in CPyCppyy::Pythonize (pyclass=0x555558bf25d0, name="std::vector<int>") at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx:1106
#34 0x00007ffff7553f74 in CPyCppyy::CreateScopeProxy (name="std::vector<int>", parent=0x555556b95eb0) at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/ProxyWrappers.cxx:686
#35 0x00007ffff7553aae in CPyCppyy::CreateScopeProxy (name="vector<int>", parent=0x0) at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/ProxyWrappers.cxx:598
#36 0x00007ffff753146d in (anonymous namespace)::MakeCppTemplateClass (args=0x7fffeb472758) at ../../../../../root_dbg_includepaths/bindings/pyroot/cppyy/CPyCppyy/src/CPyCppyyModule.cxx:368
#37 0x000055555564bc73 in PyEval_EvalFrameEx ()
#38 0x0000555555642866 in PyEval_EvalCodeEx ()
#39 0x000055555565ff55 in ?? ()
#40 0x00005555556787a4 in ?? ()
#41 0x0000555555626883 in PyObject_Call ()
#42 0x00005555556da817 in ?? ()
#43 0x0000555555626883 in PyObject_Call ()
#44 0x000055555564a502 in PyEval_EvalFrameEx ()
#45 0x0000555555642866 in PyEval_EvalCodeEx ()
#46 0x00005555556421f9 in PyEval_EvalCode ()
#47 0x0000555555674e2f in ?? ()
#48 0x000055555566fd20 in PyRun_FileExFlags ()
#49 0x000055555566f6ca in PyRun_SimpleFileExFlags ()
#50 0x0000555555610188 in Py_Main ()
#51 0x00007ffff7a3f09b in __libc_start_main (main=0x55555560fbb0 <main>, argc=2, argv=0x7fffffffe4a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe498) at ../csu/libc-start.c:308
#52 0x000055555560faea in _start ()

@eguiraud
Copy link
Member Author

cling-only repro (PyROOT is off the hook :) ):

void foo() {
   std::cout << gInterpreter->GetIncludePath() << std::endl;
   gInterpreter->Load("/home/eguiraud/ROOT/install_dbg_includepaths/lib/libvectorDict.so");
   std::cout << gInterpreter->GetIncludePath() << std::endl;
}

@eguiraud
Copy link
Member Author

Looks like it's a problem in the dictionary generation. The following snippet comes from G__vectorDict.cxx:

    static const char* includePaths[] = {                                                                               
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
0        
    };  

@vepadulano
Copy link
Member

@vepadulano can you try with makefiles instead of ninja? I don't see any other difference

Just finished compiling with cmake -GUnix\ Makefiles -Ddistcc=ON -Ddev=ON -DCMAKE_BUILD_TYPE=Debug -Dtesting=ON -Droottest=ON -DCMAKE_INSTALL_PREFIX=$ROOT_INSTALL/${PWD##*/} $ROOT_HOME and still get the same result

vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: source $ROOT_INSTALL/debugmakefiles/bin/thisroot.sh
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: which root
~/Programs/rootproject/rootinstall/debugmakefiles/bin/root
vpadulan@fedorathinkpad-T550 [~/Projects/rootcode]: python reproducer_7108.py 
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/include/" -I"/usr/include/python3.8"
-I$ROOTSYS/include -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc/" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/etc//cling" -I"/home/vpadulan/Programs/rootproject/rootinstall/debugmakefiles/include/" -I"/usr/include/python3.8"

@vepadulano
Copy link
Member

As for the includePaths variable in G__vectorDict.cxx of the build dir, here is what I get with the three different builds I've used so far

In rootbuild/debugmakefiles/core/clingutils/G__vectorDict.cxx

    static const char* includePaths[] = {
0

In rootbuild/devdebugtest/core/clingutils/G__vectorDict.cxx

    static const char* includePaths[] = {
0

In rootbuild/devrelease/core/clingutils/G__vectorDict.cxx

    static const char* includePaths[] = {
0

@eguiraud
Copy link
Member Author

As far as I can tell those includes end up in the dictionary because they are passed to TModuleGenerator::ParseArgs from RootClingMain at rootcling_impl.cxx:4491 (modGen.ParseArgs(pcmArgs)), which saves them in TModuleGenerator::fCompI and later writes them to the dictionary when TModuleGenerator::WriteRegistrationSourceImpl is called.

RootClingMain adds those includes to pcmArgs at rootcling_impl.cxx:4097 and following lines. They originally come from clingArgs, which gets them from the commandline arguments.

I don't see a way this could not happen, so figuring out what's different in Vincenzo's case should be interesting.

@eguiraud
Copy link
Member Author

@vgvassilev @Axel-Naumann ping!

@vgvassilev
Copy link
Member

It may make sense to register such include paths as "private" and adjust the interface of gSystem->GetIncludePath (cc:pcanal).

In principle we do not need this injected include path when runtime_cxxmodules are on and we might just drop that part from dictionary generation. There might be some code out there that was relying on this gray area include path.

I'd defer that to @Axel-Naumann.

@eguiraud
Copy link
Member Author

eguiraud commented Feb 1, 2021

We've been trying to figure out why Vincenzo doesn't see this. It turns it's regexes string matching: his build path starts with his source path, and that makes rootcling_impl drop buildpath/include from the list of header paths that are written to the dictionary. Offending logic is here:

std::vector<std::string> pcmArgs;
for (size_t parg = 0, n = clingArgs.size(); parg < n; ++parg) {
auto thisArg = clingArgs[parg];
auto isInclude = ROOT::TMetaUtils::BeginsWith(thisArg,"-I");
if (thisArg == "-c" ||
(gOptNoIncludePaths && isInclude)) continue;
// We now check if the include directories are not excluded
if (isInclude) {
unsigned int offset = 2; // -I is two characters. Now account for spaces
char c = thisArg[offset];
while (c == ' ') c = thisArg[++offset];
auto excludePathsEnd = gOptExcludePaths.end();
auto excludePathPos = std::find_if(gOptExcludePaths.begin(),
excludePathsEnd,
[&](const std::string& path){
return ROOT::TMetaUtils::BeginsWith(&thisArg[offset], path);});
if (excludePathsEnd != excludePathPos) continue;
}
pcmArgs.push_back(thisArg);

We think the fix should be put in ROOT_GENERATE_DICTIONARY, that already sets some excludePaths e.g. at

set(excludepaths ${CMAKE_SOURCE_DIR} ${CMAKE_BINARY_DIR}/ginclude ${CMAKE_BINARY_DIR}/externals ${CMAKE_BINARY_DIR}/builtins)

and

set(excludepaths ${CMAKE_SOURCE_DIR} ${CMAKE_BINARY_DIR})

@bendavid
Copy link
Contributor

bendavid commented Feb 2, 2021

Adding ${CMAKE_BINARY_DIR}/include to line 329 fixes the problem we were having with this when building root into a singularity image at least.

@bendavid
Copy link
Contributor

Some insight into when/where this actually causes a problem.

Take for example a recent CMSSW release where we indeed end up with a bunch of stuff in the include path starting with "/data/cmsbld/jenkins/...". If I try to do anything with gSystem->CompileMacro then things are fine if /data/cmsbld doesn't exist on the machine where I'm running, but if I create that directory and set permissions to make it inaccessible then I get a permission denied error when compiling a macro.

This also explains why I was having problems with this specifically in my singularity images, because in this case the build path was in /root/root_build, where /root still exists but is inaccessible by normal users in the final singularity image.

@vgvassilev
Copy link
Member

Looks like it's a problem in the dictionary generation. The following snippet comes from G__vectorDict.cxx:

    static const char* includePaths[] = {                                                                               
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
"/home/eguiraud/ROOT/build_dbg_includepaths/include",                                                                   
0        
    };  

Libraries are free to use the interpreter during their static initialization after being loaded. In that case this library decided to add an extra include path. If we do not need this include path (eg there is no useful header to resolve from there). In fact we ignore such include paths here. What is build_dbg_includepaths? Is it your build folder?

@eguiraud
Copy link
Member Author

eguiraud commented Nov 13, 2021

Hi @vgvassilev , I don't have the repro set up anymore, sorry

Ah but yes, build_dbg_includepaths was my build directory, that's the problem.

As Josh mentions above adding ${CMAKE_BINARY_DIR}/include to the excluded paths at the code location you mention fixes the problem.

@Axel-Naumann
Copy link
Member

@vgvassilev

In principle we do not need this injected include path when runtime_cxxmodules are on and we might just drop that part from dictionary generation.

Isn't the recent nlohmann_json issue contradicting this? These headers must be found at runtime, and that's happening either because they are found where they were during build time, or due to ROOT_INCLUDE_PATH. Unless "this" in "we do not need this injected include path" specifically refers to the ROOT build directory. I suppose we'd "survive" that because we anyway inject $ROOTSYS/include (with ROOTSYS really being "install include dir or parent dir of location of libCore") at runtime.

@vgvassilev
Copy link
Member

@Axel-Naumann, yeah, "this" means "ROOT build directory". I suspect that we did not capture properly which is the build directory. I remember some discussion about this with @linev where he reported he could not drop the root_build/include iirc.

@linev
Copy link
Member

linev commented Nov 15, 2021

I do not remember all details, but one reason why root_build/include may be still required - builtins libraries. Includes of that builtins installed in root_build/include directory and must be there when ROOT is running.

For instance - root_build/include/vdt directory which is used by RVec.hxx.
Or root_build/include/VecCore, which are used by Math/types.h

@linev
Copy link
Member

linev commented Nov 16, 2021

But maybe one can resolve this - root_build/include as include path required only during ROOT compilation.
But should not be inserted into generated ROOT dictionaries - $ROOTSYS/include anyway accessible during ROOT running.
No idea how this can be done.

@eguiraud
Copy link
Member Author

Ping :) it would be nice to fix this for v6.26

@Axel-Naumann Axel-Naumann added this to the 6.26/00 milestone Jan 5, 2022
Axel-Naumann added a commit to Axel-Naumann/root that referenced this issue Jan 10, 2022
…clude).

We do not want to remember the build directory, see issue root-project#7108.
Axel-Naumann added a commit that referenced this issue Jan 10, 2022
…clude).

We do not want to remember the build directory, see issue #7108.
Axel-Naumann added a commit to Axel-Naumann/root that referenced this issue Jan 11, 2022
…clude).

We do not want to remember the build directory, see issue root-project#7108.

(cherry picked from commit c2f028f)
Axel-Naumann added a commit that referenced this issue Jan 11, 2022
…clude).

We do not want to remember the build directory, see issue #7108.

(cherry picked from commit c2f028f)
@Axel-Naumann
Copy link
Member

Should be fixed in master and v6-26-patches. Can you confirm?

Is this also needed in 6.24?

@bendavid
Copy link
Contributor

this doesn't fix it for me unfortunately

root -l testtref.cpp+
root [0] 
Processing testtref.cpp+...
Info in <TUnixSystem::ACLiC>: creating shared library /home/b/bendavid/wmassdev/./testtref_cpp.so
cc1plus: error: /var/aurbuild/root-git/src/build/include: Permission denied
cc1plus: error: /var/aurbuild/root-git/src/build/include/: Permission denied
/usr/sbin/ld: cannot find /home/b/bendavid/wmassdev/testtref_cpp_ACLiC_dict.o: No such file or directory
collect2: error: ld returned 1 exit status
Error in <ACLiC>: Executing 'cd "/home/b/bendavid/wmassdev" ; c++ -fPIC -c -O3 -DNDEBUG -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -DIS_RPATH_BUILD=1 -pthread -g -fvar-tracking-assignments -fdebug-prefix-map=/var/aurbuild/root-git/src=/usr/src/debug -std=c++17 -Wno-implicit-fallthrough -Wno-noexcept-type -pipe -W -Woverloaded-virtual -fsigned-char -pthread  -I/usr/include -I"/etc/root" -I"/etc/root/cling" -I"/etc/root/cling/plugins/include" -I"/usr/include" -I"/var/aurbuild/root-git/src/build/include" -I"/var/aurbuild/root-git/src/build/include/"   -D__ACLIC__ "/home/b/bendavid/wmassdev/testtref_cpp_ACLiC_dict.cxx" ; c++ -O3 -DNDEBUG "/home/b/bendavid/wmassdev/testtref_cpp_ACLiC_dict.o" -shared   "/usr/lib/libm.so" "/usr/lib/libdl.so" "/usr/lib/libc.so" "/usr/lib/libgcc_s.so" "/usr/lib/libstdc++.so" "/usr/lib/root/libRint.so" "/usr/lib/root/libCore.so" "/usr/lib/libpcre.so.1" "/usr/lib/liblzma.so.5" "/usr/lib/libxxhash.so.0" "/usr/lib/liblz4.so.1" "/usr/lib/libz.so.1" "/usr/lib/libzstd.so.1" "/usr/lib/libpthread.so.0" "/usr/lib/libnss_files.so.2" "/usr/lib/root/libRIO.so" "/usr/lib/root/libThread.so" "/usr/lib/libtbb.so.12" "/usr/lib/root/libCling.so" "/usr/lib/libncursesw.so.6" "/usr/lib/root/libMathCore.so" "/usr/lib/root/libImt.so" "/usr/lib/root/libMultiProc.so" "/usr/lib/root/libNet.so" "/usr/lib/libssl.so.1.1" "/usr/lib/libcrypto.so.1.1" "/usr/lib/libnss_systemd.so.2" "/usr/lib/librt.so.1" "/usr/lib/libcrypt.so.2" "/usr/lib/libp11-kit.so.0" "/usr/lib/libffi.so.8" -o "/home/b/bendavid/wmassdev/./testtref_cpp.so"' failed!

@eguiraud
Copy link
Member Author

Uh, disappointing. My repro at the top of the issue seems to be fixed:

/tmp python
Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ROOT
>>> print(ROOT.gSystem.GetIncludePath())
-I$ROOTSYS/include -I"/home/blue/ROOT/master/install/etc/" -I"/home/blue/ROOT/master/install/etc//cling" -I"/home/blue/ROOT/master/install/etc//cling/plugins/include" -I"/home/blue/ROOT/master/install/include/" -I"/usr/include/python3.10"
>>> ROOT.vector('int')()
<cppyy.gbl.std.vector<int> object at 0x55a00ceb0a00>
>>> print(ROOT.gSystem.GetIncludePath())
-I$ROOTSYS/include -I"/home/blue/ROOT/master/install/etc/" -I"/home/blue/ROOT/master/install/etc//cling" -I"/home/blue/ROOT/master/install/etc//cling/plugins/include" -I"/home/blue/ROOT/master/install/include/" -I"/usr/include/python3.10"

(no build directory in the include paths anymore)

@bendavid
Copy link
Contributor

I think you can easily reproduce my error by keeping the build directory present but making it inaccessible by permissions.

@eguiraud
Copy link
Member Author

eguiraud commented Jan 11, 2022

I can't, sorry. I did the following:

As root user:

$ chown -R root:root cmake-build-dir
$ chmod -R o-rwx cmake-build-dir

As my user:

$ cd cmake-build-dir
cd: Permission denied: “cmake-build-dir”
$ ls cmake-build-dir
ls: cannot open directory 'cmake-build-dir': Permission denied

And still:

$ source ~/ROOT/master/install/bin/thisroot.fish
$ cat foo.cpp
#include <memory>
#include <ROOT/RDataFrame.hxx>
#include <vector>

void foo() {
      ROOT::RDataFrame df(10);
      std::vector<int> v;
}

$ root -l foo.cpp+
root [0]
Processing foo.cpp+...
Info in <TUnixSystem::ACLiC>: creating shared library /tmp/./foo_cpp.so
root [1]

@bendavid
Copy link
Contributor

I also can't reproduce this in a standalone build, only happens when I install using the system package manager (in this case archlinux in a podman image). Maybe it has to do with $ROOTSYS being set vs not.

Let me see if I can provoke it in a more standalone manner.

On the other hand if I grep for the build directory, it does still appear in the final .so and .pcm files.

@lmoneta lmoneta removed this from the 6.26/00 milestone Jan 24, 2022
rahulgrit pushed a commit to rahulgrit/root that referenced this issue Jan 25, 2022
…clude).

We do not want to remember the build directory, see issue root-project#7108.
@eguiraud
Copy link
Member Author

On the other hand if I grep for the build directory, it does still appear in the final .so and .pcm files.

That might or might not be a problem, I guess?

With the original reproducer fixed I'm not sure how to make progress here :/

@eguiraud
Copy link
Member Author

eguiraud commented Jun 2, 2023

Closing since the original reproducer has been fixed.

@eguiraud eguiraud closed this as completed Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment