Skip to content

Advanced Git Topics

Jump to bottom Edit New page
Daniel Peter edited this page Nov 15, 2023 · 2 revisions

FOR ADVANCED USERS ONLY

PLEASE DO NOT USE UNLESS YOU KNOW WHAT YOU ARE DOING

Table of Contents

FOR ADVANCED USERS ONLY

PLEASE DO NOT USE UNLESS YOU KNOW WHAT YOU ARE DOING

Git Workflow

Introduction

The workflow we are using follows the Integration-Manager Workflow. This document summarizes this workflow and describes the specifics of using it on Github. This is also referred to as the Fork&Pull Model. This is an adaptation of the gitflow model which is based on the original article by Vincent Driessen and used for instance in the petsc project.

Repositories

We are using GitHub to host our git repositories. There are three types of repositories:

  • Repositories hosted on GitHub:

    • The central (blessed) repository.

    • The developers's public repositories.

  • Each developer will have their own private repositories hosted on their local machines

Users and Permissions

They are two sets of users of the repositories: maintainers and developers. What differentiates them is their permissions on the central repository:

  • Maintainers have push-pull access.

  • Developers only have pull access.

Initial Set up for Developers

Assuming that the central repository has been set up, the first time a developer wants to contribute he or she needs to set up their GitHub and private repositories with these two steps (See Fork A Repo):

  1. Fork the central repository to create his or her GitHub repository.

  2. Clone his or her GitHub repository locally on their own machines.

Typical Process for Contributing Code

After the initial set up, a typical process for a developer to contribute code is (see Integration-Manager workflow in Pro Git book).

  1. The contributor fetches the latest changes from the central repository to their local, private repository.

  2. The contributor makes changes on their private repository.

  3. The contributor pushes the changes to their GitHub repository (See Pushing to a remote).

  4. The contributor initiates a pull-request for their changes.

  5. Our buildbot automatically pulls the changes and tests them automatically.

  6. If the changes pass the tests, the maintainer accept the changes and they are committed to the central repository otherwise the contributor is asked to fix the problems.

Git Tips For developers

Git repository setup

This describes how to follow our workflow using git instead of hub. The, simpler, procedure with hub is described in Using Git for SPECFEM through hub.

The examples use a fictitious user named John Doe whose github user name is johndoe.

Git set up

See Git Book - Getting Started - First-Time Git Setup.

From your local machine, tell git who you are (see ~/.gitconfig)

$ git config --global user.name "John Doe"
$ git config --global user.email john.doe@gmail.com

note: this is only necessary once

Fork a repository

This is done in a web browser, see (https://help.github.com/articles/fork-a-repo).

Clone your fork locally

$ git clone https://github.com/johndoe/specfem3d.git SPECFEM3D

Configure remotes connecting to the original repository

$ cd SPECFEM3D
$ git remote add upstream https://github.com/SPECFEM/specfem3d.git

$ git fetch upstream
From https://github.com/SPECFEM/specfem3d
 * [new branch]      coupling_vadim -> upstream/coupling_vadim
 * [new branch]      devel      -> upstream/devel
 * [new branch]      master     -> upstream/master
 * [new branch]      update_temporary -> upstream/update_temporary

Setup a tracking branch for the devel branch of your fork

This is because when you clone a repository, it generally automatically creates a master branch that tracks origin/master (see Git Book - Git Branching - Remote branching). You need to set up a tracking branch for devel:

$ git checkout --track origin/devel

Develop new contributions

Make the changes

  • Check that you are in your devel branch

    $ git branch
* devel
master

  • Modify the code.

    $ cd SPECFEM3D/utils/scripts/Cluster/pbs/

  • Edit some file.

  • Then check status

    $ git status
# On branch devel
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   go_generate_databases_pbs.bash
#
no changes added to commit (use "git add" and/or "git commit -a")

Commit your changes locally

$ git add go_generate_databases_pbs.bash
$ git commit -a -m "Change first line of go_generate_databases_pbs.bash"
[feature24 f618395] Change first line of go_generate_databases_pbs.bash
 1 file changed, 2 insertions(+), 2 deletions(-)

Note: You can use " git commit -a " without the -m option to ask Git to open an editor for you, in which you can type your commit message (very useful if the message is long and/or if you want to type several lines for instance). The editor used is the one that is defined by the EDITOR variable in your .bashrc or .cshrc file. (e.g. setenv EDITOR vim or similar)

Note: You can use " git commit -a –amend " to fix commits that included accidental changes. (Just do not do it on commits that are already merged to devel.) Beware that git commit –amend on the merge commit will not work correctly. First, being a merge, it really shouldn't contain "extra" changes besides the merging. Second, you shouldn't amend commits that are already public (or at least, in the main repo).

$ git status
# On branch devel
nothing to commit, working directory clean

Check that your local repository is in sync with upstream

  • Fetch the changes

    $ git fetch upstream

  • See what has changed

    $ git diff devel upstream/devel

  • Merge the potential changes

    $ git branch
* devel
 master
$ git merge upstream/devel

  • Push your development to your forked GitHub repository

    $ git push origin devel
Counting objects: 22, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 503 bytes | 0 bytes/s, done.
Total 6 (delta 5), reused 0 (delta 0)
To git@github.com:johndoe/specfem3d.git
   4ba0960..f618395  devel -> devel

Submit new contributions upstream (SPECFEM)

Make a pull request from your forked GitHub repository page

  • Go to your forked repository

  • Click "Pull Request"

  • Click "New Pull Request"

  • Create a pull-request to push the changes from johndoe:devel into SPECFEM:devel Note that you are not pushing the johndoe:devel branch into the SPECFEM/devel branch but into SPECFEM:devel.

  • Check that the message says "Able to merge - These branches can be automatically merged"

  • Click "Click to create pull request for this comparison".

Await decision from the code maintainers

Current SPECFEM3D code maintainers are: Matthieu, Dimitri, Daniel and David.

Keeping your repository in sync

Unless you are using feature branches (which are not described here), you should probably wait until your changes are merged before updating your local repository. This will avoid generating unnecessary merges or alternate commits. If you are familiar with git, you likely already know how to work around these limits.

Updating your local repository is a three stage process:

  • Fetch new commits from upstream (the SPECFEM repository):

    $ git fetch upstream
remote: Counting objects: 91, done.
remote: Compressing objects: 100% (75/75), done.
remote: Total 91 (delta 48), reused 34 (delta 16)
Unpacking objects: 100% (91/91), done.
From https://github.com/SPECFEM/specfem3d
   f36f97e..b87b2a2  devel      -> upstream/devel
Fetching submodule EXAMPLES
remote: Counting objects: 152, done.
remote: Compressing objects: 100% (85/85), done.
remote: Total 85 (delta 73), reused 0 (delta 0)
Unpacking objects: 100% (85/85), done.
From git://vcs.geodynamics.org/specfem3d_examples
   a87764f..bf38c5d  master     -> origin/master

    You may not see any updates to the submodules if they have not been changed. (The above sample shows an update to the examples.)

  • Fast-forward merge your repository branch with the upstream branch:

    $ git checkout devel # Not necessary if you're already on devel.
$ git merge --ff-only upstream/devel
Updating f36f97e..b87b2a2
Fast-forward
 DATA/Par_file                                    |    6 +-
 EXAMPLES                                         |    2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)
$ git submodule update # Only necessary if submodules have changed
Submodule path 'EXAMPLES': checked out 'bf38c5d4415ed828f173d493adafc1f07c31e7c2'

    You can also apply the above steps to the master branch if you wish. As the master branch is updated very slowly, it is unlikely you will need to do so very often.

    If git throws an error about being unable to fast-forward merge, then you went against recommendations here and made additional changes. There are two possibilities:

    • You modified a few files, but did not commit them. You can use the command git stash to temporarily store your changes, then git stash pop at the end of these instructions to restore the changes.

    • You modified files and made additional commits. Removing the --ff-only flag will allow the merge to proceed, but you may need to resolve conflicts and you will be creating a merge commit. Be sure that is your intended result.

  • Push your local changes to your fork on GitHub:

    $ git push origin devel
Counting objects: 230, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (151/151), done.
Writing objects: 100% (162/162), 77.73 KiB | 0 bytes/s, done.
Total 162 (delta 120), reused 22 (delta 11)
To git@github.com:johndoe/specfem3d.git
   3d0000c..b87b2a2  devel -> devel

    This step is not strictly necessary. You are free to wait until you have made additional changes before pushing to GitHub.

Committing only part of your local changes

Git has ways to help with these cases. You can use git add -p to selectively add changes to commit and git commit (no -a) to commit only those changes. That allows you to split changes into logical chunks. Another way, if you're working on something and find a bug, you can use git stash to temporarily store your changes, fix the bug and commit it, then git stash pop to restore your original work.

Useful Aliases

Some developers find it very useful to define aliases once and for all in their .bash_profile or .cshrc file for the most useful sets of Git commands to type, and then use them all the time instead of the numerous Git commands above. Here are examples for a .cshrc file:

  • To get a copy of the "devel" branch of the code once and for all (replace "komatits" with your login at GitHub):

    alias getinitialcopyofthecode 'git clone --recursive https://github.com/komatits/specfem3d.git SPECFEM3D ; cd SPECFEM3D/ ; git remote add upstream https://github.com/SPECFEM/specfem3d.git ; git fetch upstream ; git remote set-url origin https://komatits@github.com/komatits/specfem3d.git ; git checkout --track origin/devel ; cd examples ; git checkout master ; git remote set-url --push origin ssh://git@vcs.geodynamics.org/specfem3d_examples ; cd .. ; git branch '

  • To commit all your modified files:

    alias gitcommit "git commit -a"

  • To push all your local (committed) changes to your GitHub fork and then merge them to the official Git version of SPECFEM using a pull request (this does NOT apply to the "examples" directory because it is not stored at GitHub, see below):

    alias gitpush "git push origin devel ; echo 'now go to the GitHub web site to do a pull request'"

  • To push all your local (committed) changes to the examples:

    alias gitpushexamples "git push origin master"

  • To keep your local version of the code up-to-date:

    alias gitupdate "git fetch upstream ; git checkout devel ; git merge --ff-only upstream/devel ; git submodule update ; git push origin devel"

  • And then, to work on the code, all you will ever have to do is type:

    getinitialcopyofthecode     (once and for all, to get the initial copy of the code after creating a "Fork" on your GitHub web page)

  • and then, in everyday work:

    gitupdate

gitcommit

gitpush

Note (important): Note also that many users find it very useful / crucial to install a high-level GUI interface to GIT, for instance SmartGit ( http://www.syntevo.com/smartgithg ).

GUI for git

Git References

Development wiki for SPECFEM3D

Development wiki

User manual

Clone this wiki locally