Git Repository Structure

Repository structure

• seqan/seqan3
  • the library
  • api reference (inside library code)
  • cmake file and pkgconfig file
• seqan/seqan3/wiki (wiki page of seqan/seqan3)
  • "SeqAn project" organisation, e.g. who-is-who, contributor guides...
  • resources for developers of the library, e.g. release check-lists
• seqan/seqan3-manual
  • readthedocs stuff, but only for users of the library, not developers of the library
• seqan/seqan3-test
  • unit tests
• seqan/seqan3-util
  • any other stuff, e.g. additional cmake files, application templates...

Apps go wherever they want to, completely unrelated.

seqan/seqan3-test

The idea is to add seqan3 as a submodule to the seqan3-test repo. There will also be a submodule to either google test or boost test. It turned out, that decoupling the tests from the source through different git repositories can lead to substantial damage and nightmares of conflict resolutions. The better approach would be to develop tests and sources together in one repository. Here is the proposed structure of the repositories:

seqan/seqan3/
            - include/
                     - seqan3/...   [**our headers**]
                     - sdsl/...     [**sdsl submodule/subtree?**]
                     - cereal/...   [**cereal submodule/subtree?**]
                     - range-v3/... [**range-v3 submodule/subtree?**]
            - tests/...             [**unit tests**]
            - CMakeLists.txt
            - CHANGELOG.md
            - CONTRIBUTING.md
            - LICENSE
            - README.md

or

seqan/seqan3/
            - contrib/ 
                     - sdsl/...     [**sdsl submodule/subtree?**]
                     - cereal/...   [**cereal submodule/subtree?**]
                     - range-v3/... [**range-v3 submodule/subtree?**]
            - include/seqan3/...    [**our headers**]
            - tests/...             [**unit tests**]
            - CMakeLists.txt
            - CHANGELOG.md
            - CONTRIBUTING.md
            - LICENSE
            - README.md

The first structure makes somewhat clear, that we are using these submodules as hard requirements, so seqan3 won't be functioning properly without these projects. The second structure makes more clear, that we require some contributions from other sites, but our include only focus on seqan code. At the moment I tend to use the first structure.

Now the question whether to use git submodule or git subtree. The pro's and con's are very subtle I think, but here is my reasoning for git submodules rather than git subtrees. From the user handling and simplicity I would prefer the git subtree version, but the biggest issue is that we will copy the entire repository with all the clutter utility stuff, that we just wanted to get rid off. So if we can assume, that we merely pull changes from the contrib repos, and do not much of upstream contribution, than we are quite save with the submodule structure. The only downside is, that people have to clone it with --recursive flag, but I think we can stand that, since most people can use one of the system's package manager to install seqan and then they don't have to worry about the dependencies not pulled completely.

Now let's see how we could structure the seqan3-test repository.

seqan/sean3-test/
                - google-test [**submodule**]
                - seqan3      [**submodule**]
                - seqan3-util [**submodule**] 
                - CMakeLists.txt
                - CHANGELOG.md
                - CONTRIBUTING.md
                - LICENSE
                - README.md

seqan3-util comes with all the build system stuff and cmake-config files, which will also be used by the applications pulling in seqan3 as a submodule. Hence we can externalise it into a separate repository. In addition we pull the test framework. In this case the google-test, which is first build as static library and then linked against when building the test cases.

old (maybe reuse it for some other stuff)

Every developer has to fork the seqan3 repository and the seqan3-test repository into their own organisation. Then the seqan3-test repository has to be cloned with the --recursive flag.

cd $HOME
git clone --recursive git@github.com:my_organisation/seqan3-test.git

This will clone and initialise all submodules included into the seqan3-test repository. There will be a master and a develop branch for both the seqan3 and the seqan3-test repository. The seqan3 repository merely contains the sources, a readme, a contributor and a license file as well as a CMakeLists.txt file for configuring the project. The entire build stuff should go into the separate repository seqan3-util.

Now let's see how we can develop a new feature. We strongly emphasise to go through the Test-Driven-Development (TDD) approach. Since we use git submodule the tests are not coupled with the sources anymore which can lead to disambiguities in the commit history, e.g. the latest commit of the test module does not reflect the proper state of the submodule.

To prepare our local working copy we first have to add the remote for our forked repository for seqan3-test and for the contained submodule.

cd seqan3-test
git remote add my_origin git@github.com:my_organisation/seqan3-test.git
git fetch my_origin
cd include/seqan3
git remote add my_origin git@github.com:my_organisation/seqan3.git
git fetch my_origin

If we want to develop a new feature we create a new branch from develop in the local working copy of our seqan3-test fork.

cd ../..
git checkout -b feature/my_feature my_origin/develop

Now we can add a test in the proper directory. To make our test run successfully, we need to add a new feature into seqan3. So we go into seqan3 and add a new feature branch from develop.

cd include/seqan3
git checkout -b feature/my_feature my_origin/develop

We give the feature branch the same name as we used for the test case. When the feature is ready and the tests pass, we can create a new pull request against the repository.