Skip to content

Commit

Permalink
[cxxmodules] Fix failing runtime_cxxmodules tests by preloading modules
Browse files Browse the repository at this point in the history
Currently, 36 tests are failing for runtime modules:
https://epsft-jenkins.cern.ch/view/ROOT/job/root-nightly-runtime-cxxmodules/
We want to make these test pass so that we can say runtime modules is
finally working.

This patch enables ROOT to preload all modules at startup time. In my
enviroment, this patch fixes 14 tests for runtime cxxmodules.

Preloading all the modules has several advantages. 1. We do not have to
rely on rootmap files which doesn't support some features (namespaces and
    templates) 2. Lookup would be faster because we don't have to do
trampolin via rootmap files.

The only disadvantage of preloading all the modules is the start up time performance.
`root.exe -q -l memory.C`
This is a relase build without modules:
```
 cpu  time = 0.091694 seconds
 sys  time = 0.026187 seconds
 res  memory = 133.008 Mbytes
 vir  memory = 217.742 Mbytes
```
This is a release build with modules, with this patch:
```
 cpu  time = 0.234134 seconds
 sys  time = 0.066774 seconds
 res  memory = 275.301 Mbytes
 vir  memory = 491.832 Mbytes
```
As you can see, preloading all the modules makes both time and memory 2
to 3 times worse **at startup time**.

However, it is a matter of couse that we get slower startup time if we
try to load modules at start up time, not on-demand. I haven't had the
good benchmark for this but in theory it reduces execution time instead,
     because currently we're anyway loading modules after
     the startup.
  • Loading branch information
yamaguchi1024 committed Apr 4, 2018
1 parent f688d41 commit fb53571
Showing 1 changed file with 42 additions and 1 deletion.
43 changes: 42 additions & 1 deletion core/metacling/src/TCling.cxx
Original file line number Diff line number Diff line change
Expand Up @@ -1114,6 +1114,22 @@ static bool IsFromRootCling() {
return foundSymbol;
}

static std::string GetModuleNameAsString(clang::Module *M, const clang::Preprocessor &PP)
{
const HeaderSearchOptions &HSOpts = PP.getHeaderSearchInfo().getHeaderSearchOpts();

std::string ModuleFileName;
if (!HSOpts.PrebuiltModulePaths.empty())
// Load the module from the prebuilt module path.
ModuleFileName = PP.getHeaderSearchInfo().getModuleFileName(M->Name, "", /*UsePrebuiltPath*/ true);
if (ModuleFileName.empty()) return "";

std::size_t found = ModuleFileName.find_last_of("/\\");
std::string ModuleName = ModuleFileName.substr(found + 1);
ModuleName.erase(ModuleName.end() - 4, ModuleName.end());
return ModuleName;
}

////////////////////////////////////////////////////////////////////////////////
/// Initialize the cling interpreter interface.

Expand Down Expand Up @@ -1258,7 +1274,32 @@ TCling::TCling(const char *name, const char *title)
// Load libc and stl first.
LoadModules({"libc", "stl"}, *fInterpreter);

LoadModules({"ROOT_Foundation_C", "ROOT_Config", "ROOT_Foundation_Stage1_NoRTTI", "Core", "RIO"}, *fInterpreter);
// Load core modules
std::vector<std::string> CoreModules = {"ROOT_Foundation_C","ROOT_Config","ROOT_Foundation_Stage1_NoRTTI", "Core", "RIO"};
LoadModules(CoreModules, *fInterpreter);

// Otherwise module conflicts between rootcling and root
if (!fromRootCling) {
// Dynamically get all the modules and load them if they are not in core modules
clang::CompilerInstance &CI = *fInterpreter->getCI();
clang::ModuleMap &moduleMap = CI.getPreprocessor().getHeaderSearchInfo().getModuleMap();
clang::Preprocessor &PP = CI.getPreprocessor();
std::vector<std::string> vec;

for (auto I = moduleMap.module_begin(), E = moduleMap.module_end(); I != E; ++I) {
clang::Module *M = I->second;
assert(M);

std::string ModuleName = GetModuleNameAsString(M, PP);
if (!ModuleName.empty() && std::find(CoreModules.begin(), CoreModules.end(), ModuleName) == CoreModules.end()) {
if (M->IsSystem && !M->IsMissingRequirement)
LoadModule(ModuleName, *fInterpreter);
else if (!M->IsSystem && !M->IsMissingRequirement)
vec.push_back(ModuleName);
}
}
LoadModules(vec, *fInterpreter);
}

// Check that the gROOT macro was exported by any core module.
assert(fInterpreter->getMacro("gROOT") && "Couldn't load gROOT macro?");
Expand Down

0 comments on commit fb53571

Please sign in to comment.