External subprojects using Git and CMake

Git offers the possibility of splitting one project into many subprojects, each living in its own repository and undergoing development separately. The superproject tracks a single commit (not a branch!) for each of the subprojects. Whenever a subproject is updated, the superproject maintainer can decide to update it or not. Here you can find a nice tutorial on submodules.

This gives a lot of flexibility, e.g. when developing a new interface with an external library. It is especially convenient when coupled to the external project module of CMake.

Adding submodules as CMake external projects

First of all, we have to set up the submodule on the Git side. Let us take the PCMSolver project as an example. The project is hosted at https://github.com/PCMSolver/pcmsolver and we want to clone it into the external/pcmsolver directory:

git submodule add https://github.com/PCMSolver/pcmsolver external/pcmsolver

Issuing a git status, we will notice:

On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

   new file:   .gitmodules
   new file:   external/pcmsolver

We can now git commit and half of the work is done. Git now knows how to fetch the external project. It remains to instruct CMake on how to configure and build it. Depending on how the external module should interact with Psi you will need to add some lines to one or more CMakeLists.txt file. In the case of our example PCMSolver, we need to modify the root CMakeLists.txt and the one in src/lib/libpcm. The additions to the root CMakeLists.txt look like:

if(ENABLE_PCMSOLVER)
   find_package(ZLIB)
   if(NOT ZLIB_FOUND)
      set(ENABLE_PCMSOLVER OFF)
      message(STATUS "Polarizable Continuum Model via PCMSolver DISABLED")
      message(STATUS " PCMSolver dependencies NOT satisfied:")
      if(NOT ZLIB_FOUND)
         message(STATUS "  - install Zlib development libraries")
      endif()
   else()
      fortran_enabler()
      add_definitions(-DHAVE_PCMSOLVER)
      message(STATUS "Polarizable Continuum Model via PCMSolver ENABLED")
      link_directories(${PROJECT_BINARY_DIR}/external/lib)
      # PCMSolver needs Fortran libraries 
      if(CMAKE_CXX_COMPILER_ID MATCHES GNU OR CMAKE_CXX_COMPILER_ID MATCHES Clang)
         set(PCMSOLVER_LIBS pcm getkw gfortran quadmath ${ZLIB_LIBRARIES})
      else() # Intel case
         set(PCMSOLVER_LIBS pcm getkw ifcore imf ${ZLIB_LIBRARIES})
      endif()
      set(PSILIB ${PSILIB} psipcm ${PCMSOLVER_LIBS})
   endif()
   set(PCMSOLVER_PARSE_DIR ${EXTERNAL_PROJECT_INSTALL_PREFIX}/bin)
endif()

The module is configured and built only it the ENABLE_PCMSOLVER variable is set, either via the setup script or passing it directly to CMake. The additional dependency on zlib introduced by the module is checked. Notice that configuration of the module is skipped if the dependency is not satisfied, a warning is printed out if that is the case. The CMakeLists.txt file in src/lib/libpcm is structured as follows. Don't forget to include(ConfigExternal):

if(ENABLE_PCMSOLVER)
   include(ConfigExternal)
   set(PCMSOLVER_TESTS ON)
   get_filename_component(ZLIB_ROOT ${ZLIB_LIBRARIES} PATH)
   # PCMSolver does not know profile
   if(CMAKE_BUILD_TYPE MATCHES "profile")
      set(PCM_BUILD_TYPE "release")
   else()
      set(PCM_BUILD_TYPE ${CMAKE_BUILD_TYPE})
   endif()
   list(APPEND PCMSolverCMakeArgs
       -DCMAKE_BUILD_TYPE=${PCM_BUILD_TYPE}
       -DCMAKE_INSTALL_PREFIX=${PROJECT_BINARY_DIR}/external
       -DCMAKE_Fortran_COMPILER=${CMAKE_Fortran_COMPILER}
       -DEXTRA_Fortran_FLAGS=${PCM_EXTRA_Fortran_FLAGS}
       -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
       -DEXTRA_C_FLAGS=${PCM_EXTRA_C_FLAGS}
       -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
       -DEXTRA_CXX_FLAGS=${PCM_EXTRA_CXX_FLAGS}
       -DENABLE_CXX11_SUPPORT=${ENABLE_CXX11_SUPPORT}
       -DPARENT_INCLUDE_DIR=${PROJECT_BINARY_DIR}/external/include
       -DPARENT_DEFINITIONS=${PARENT_DEFINITIONS}
       -DEIGEN3_ROOT=${EIGEN3_ROOT}
       -DDISABLE_EIGEN_OWN=${DISABLE_EIGEN_OWN}
       -DENABLE_EIGEN_MKL=${ENABLE_EIGEN_MKL}
       -DPARENT_BINARY_DIR=${PROJECT_BINARY_DIR}
       -DBOOST_INCLUDEDIR=${BOOST_INCLUDE_DIRS}
       -DBOOST_LIBRARYDIR=${BOOST_LIBRARIES}
       -DENABLE_64BIT_INTEGERS=${ENABLE_64BIT_INTEGERS}
       -DENABLE_TESTS=${PCMSOLVER_TESTS}
       -DZLIB_ROOT=${ZLIB_ROOT}
       -DPYTHON_INTERPRETER=${PYTHON_EXECUTABLE}
      )

   if(PCMSOLVER_TESTS)
      add_external(pcmsolver "${PCMSolverCMakeArgs}" ${CMAKE_CTEST_COMMAND})
   else()
      add_external(pcmsolver "${PCMSolverCMakeArgs}")
   endif()

   link_directories(${EXTERNAL_PROJECT_INSTALL_PREFIX}/lib
                    ${EXTERNAL_PROJECT_INSTALL_PREFIX})

   include_directories(${CMAKE_BINARY_DIR}/external/include
                       ${CMAKE_BINARY_DIR}/external/include/eigen3/include/eigen3)
   set(PCMSOLVER_PARSE_DIR ${EXTERNAL_PROJECT_INSTALL_PREFIX}/bin)
   configure_file(${CMAKE_SOURCE_DIR}/lib/python/pcm_placeholder.py.in 
                  ${CMAKE_SOURCE_DIR}/lib/python/pcm_placeholder.py)
endif()

We are here using the add_external macro (defined in cmake/ConfigExternal.cmake) to correctly set up CMake for the external module. This macro takes three arguments:

add_external(_project_name _external_project_cmake_args _testing_command)

• _project is the name of the external project. This has to be the same as the name of the subdirectory in external where the project was cloned.
• _external_project_cmake_args is a list of -D... options. These will be passed to the external module CMake system;
• _testing_command is the command that CMake will use to test the submodule before it is installed. If testing fails, Psi compilation will be aborted Submodules testing can be skipped by not passing a third argument.

Finally, make sure that:

• linking of the resulting library is properly managed in Psi CMake system;
• if any library in Psi depends on the external module, dependencies are properly handled. For example: add_dependencies(psipcm pcmsolver)

Building Psi without the submodules

This is easy. Just configure and build as you usually would. One important caveat: even if you won't be compiling the external projects, you will need to have access to their repositories. Before merging any functionality depending on external submodules to master, make that cloning it doesn't require any special action from the other developers!

Check out external sources without building Psi

This is easily done with the following combo:

git submodule init
git submodule update

You will find the sources where they were configured to be when first added to the project, most likely in external.

Update external sources

When initializing an external module, you will be referencing a specific commit. For each external project, the Git system in Psi will be pointing to this commit. This pointer will not change unless you tell Psi that it has changed, meaning that you updated the external sources and committed the change in the pointed-to-commit to Psi. To update external sources, you need to go to the directory where the external project in question was cloned:

 cd external/pcmsolver

Now you can switch to the branch you would like to reference and possibly update it:

 git checkout awesome_new_feature
 git pull origin awesome_new_feature

It is now time to register the change within Psi:

 cd ../..
 git add external/pcmsolver
 git commit -m "Update reference to PCMSolver external module."

That's it!

How to work on both projects simultaneously?

Suppose you are trying to interface Psi and an external module. You need to modify some files in Psi and some in the external module. How to do that? First of all, you will need to switch to the proper branch for the module. When any external project is cloned it is in a "detached HEAD" state, since it points to a specific commit, not a branch.

cd external/pcmsolver
git checkout psi_interface
... some coding goes on here ...
git add psi_interface.cpp
git commit -m "Added file for interfacing with psi"
git push origin psi_interface

You then have to follow a similar workflow in Psi. The first thing to do is to commit modifications to the external modules:

cd ../..
git commit external/pcmsolver
... some coding goes on here ...
git commit UsePCMSolver.h
git push origin pcm

External project tracking different remotes on different Psi branches

This sounds involved but it might happen more often than you think. Let’s again take PCMSolver as an example. The version released in Psi lives on GitHub and is publicly accessible. On some other Psi branches though, we would maybe like to work with the development version of the module. This version is not publicly accessible and resides at another remote URL.

Each Psi branch has its own .gitmodules file, correctly configured to track the right remote. When switching Psi branches though, the same switch of remotes does not happen for submodules. To be more explicit, assume you have Psi_branch-public tracking GitHub and Psi_branch-private tracking repo.ctcc.no. Moreover, assume you are currently working on Psi_branch-public. As explained here, the correct workflow to switch from one to the other is:

git checkout Psi_branch-private
git submodule sync

The git submodule sync command synchronizes submodules’ remote URL configuration setting to the value specified in .gitmodules (taken from here)

Removing submodules

Removing a submodule requires following the steps outlined here