Skip to content

git workflow

Jump to bottom
Fabian Wein edited this page Jan 4, 2021 · 52 revisions

back to home

Git Workflow used for CFS development

Getting Started

To get started set up your username and email, such that gitlab can correctly identify your commits

git config --global user.name "<your GitLab Name (can have spaces)>"
git config --global user.email "<your emai>"

A project can be cloned eiter via HTTPS or via SSH, where SSH is preferred. To clone via SSH upload your public key to gitlab once. For HTTPS you will need to enter username and password every time. The clone URL is available at every project page, e.g. for CFS and can be switched between https and ssh. To clone, open a terminal and use, e.g.

git clone --recurse-submodules https://cfs-dev.mdmt.tuwien.ac.at/cfs/CFS TARGET_DIRECTORY
or
git clone --recurse-submodules ssh://git@cfs-dev.mdmt.tuwien.ac.at:1022/cfs/CFS.git TARGET_DIRECTORY

where TARGET_DIRECTORY is optional. You now have a local copy of the repository in the directory TARGET_DIRECTORY or CFS if TARGET_DIRECTORY was not specified. If you only want to clone a single branch, use

git clone -b BRANCH_NAME --single-branch https://cfs-dev.mdmt.tuwien.ac.at/cfs/CFS TARGET_DIRECTORY
git clone -b BRANCH_NAME --single-branch ssh://git@cfs-dev.mdmt.tuwien.ac.at:1022/cfs/CFS.git TARGET_DIRECTORY

If you clone the complete repository you are on branch master by default. To list all available branches in this repository do

git branch -a

Then switch to your branch with

git checkout BRANCH_NAME

You are now on your branch which you can verify by

git status

The typical workflow with git is the following

  1. get the up to date working copy (git pull)
  2. navigate to your working copy, change stuff, test ...
  3. add files to prepare a commit (git add ...)
  4. commit the files to the local working copy (git commit ...)
  5. share your work, i.e. push to the remote repository (git push ...)

If you have made commits to your local repository, and there are modifications upstream (on the remote server), git has two options to combine both versions when you do a git pull. The default is to merge the branches, creating an unnecessary merge commit. Therefore, always pull using the --rebase option. This will put your local commits after the upstream changes. You can set this as the default by

git config [--global] pull.rebase true

Repository Management, Branches and Merge-Requests

We use a centralized workflow with the two main long running branches master and shared_opt[_0714]. To make sure nothing accidentally ends up on the master branch it is protected, thus only pepole with sufficient permissions can push/merge into it. Everything that goes into the master branch must pass all required tests defined in the pipeline.

Always base new work on top of master, and keep it updated with changes by rebase. All other branches are subject to change, do not base work on top of them!

To keep the repository tidy please

  • choose descriptive branch names, e.g. kroppert-harmonic-balance-fem or issue-NN-sequence-step-bugfix
  • assign the same branch names in the Testsuite repository (if you create tests)
  • delete branches that are not needed any more

In order to get your work into the master branch, you need to

  1. make sure it merges to the master branch in a fast-forward merge
  2. make sure it passes the pipeline
  3. create a merge request from your branch to master and assign it to a maintainer for review

We have the testing branch, which usually (if nobody is currently testing) should match the master branch. Every developer can use it to test, e.g. if a merge works without conflict. When you push to testing the pipeline will automatically run using the available build cache for the cfsdeps (faster than a complete rebuild happening for a new branch name).

Small commits to long running branches

If your fix is small and you can/want to do it in a single commit the procedure is

  1. ensure that your working copy (in this case of master) is up-to-date
  2. do your fix, commit it locally, test locally
  3. push it to the testing branch, and create a merge request (testing->master)
# assuming you start with the master branch checked out
git status
git pull
# do your fix
git diff # see what you changed
git add
git commit # write a message
# test before pushing to the remote
git push origin master:testing

Intermediate size changes

For changes which will require some work locally (e.g. fix a small bug) one will typically create a local branch:

git checkout -b bug_123

now you can try out stuff, commit locally whenever you want, and test the implementation:

git commit -a -m "fixed one thing, broke some testcases"
git commit -a -m "this works with test XXX"
git commit -a -m "all tests pass"

Once you are satisfied with the implementation, it's time to share it. You can (optionally) first re-organize your work locally (git rebase -i SHA-to-rebase-on, typically git rebase -i master, see the rebase example). Then push your branch to the server (the git remote called origin)

git push [origin [bug_123]]

where a new branch will be created.

In order to get your changes into the master branch they should be re-organized into a single commit, and fast-forward mergable on top of the master branch. Use (interactive) rebase to do that (see the rebase example). First make sure your local master branch is up-to-date, then

git checkout bug_123
git rebase -i master

Check your history with

git log

Finally, push to the testing branch to trigger a pipeline

git push origin bug_123:testing

If above push does not work (because it is not fast-forward) someone else is currently using the testing branch. You can also run the pipeline for your branch).

Short instruction sequence that proved to be useful: https://tech.serhatteker.com/post/2019-06/rebase-master/

Long running feature branches

To switch to a certain branch use, e.g. shared_opt_0714

git checkout shared_opt_0714

For branches already existing on the remote, git will create a local branch which tracks the remote branch. If the branch does not exist yet you can push it to the remote using

git push -u origin shared_opt_0714

You can now work on your feature branch.

Working with Submodules

A git-submodule is a reference to another git repository within the main repository. Git only stores the SHA and the URL of the submodule-repo. We use this for the Testsuite.

Typical errors are:

  • commit of Testsuite changes although you did not change anything, typically to an old Testsuite SHA. This is caused by git not updating submodules in the default behaviour: Always check what is staged for commit!
  • merge conflicts in the Testsuite caused by changes both in master and feature branch: Make sure your Testsuite changes are on top of master (in the Testsuite repo), then commit the correct SHA (of your Testsuite version) and add it in the CFS repo to resolve the conflict.

To update a git-submodule from the main repo run

git submodule update

To update the submodule automatically on git pull of the main repo set

git config --global submodule.recurse true

You can show differences in the submodule by (working for git >= 2.11]

git diff --submodule=diff

Sometimes above methods do not work. Then you can cd into the submodule folder and work with it like an usual git working copy:

cd Testsuite
(git checkout master)
git pull
cd ..

Examples

Creating a new testcase

  1. go to the testsuite (submodule), create the testcase, when you are ready to commit
  2. make a new branch git checkout -b my-great-test
  3. add files, commit (git add ..., git commit)
  4. test locally and push when ready
  5. go (up) to the main cfs repo and create a matching branch (cd ..; git checkout -b my-great-test) or checkout your feature branch
  6. commit the changes in the Testsuite submodule: git add Testsuite; git commit
  7. push
  8. to bring stuff to master, make merge requests for both the testsuite and the cfs master, e.g. via the web frontend.

In case master changed during working on the testsuite branch point 4 (push when ready) is more specifically

git rebase -i origin/master
git push --force origin +my-great-test

rebasing

Say you have created a few commits on your branch bug_123 and want to clean up the history: you need to rebase

  1. find out on top of which commit your changes should go (e.g. git log or some git gui), note down the ref (SHA, branch, or tag) to rebase on
  2. check out your branch git checkout bug_123 and start an interactive rebase by git rebase -i <ref-to-rebase-on>
  3. Now you will see all commits you have done, and you have to choose what to do with them. Typically you will use some of the following options:
  • squash to combine a commit it with previous ones, or
  • amend to modify the commit and/or its commit message, or
  • drop to get rid of the commit alltogether (just delete the line), or
  • you can re-order the commits (by changing the line order).
  • If you want to end with a single commit, simply select squash for all but the fist one. Finally, save the file and git will start the rebase.
  1. Whenever git stops, e.g. for amend, follow the instructions in git status until the rebase is done.

For details on rebasing see Atlassian rebase tutorial.

Squash intermediate branch to a single commit

In case your intermediate branch consists of several commits (backup, fixing, ...) and you want them to result in a single commit on the master, it is more convenient to merge instead of rebasing and squashing everything but the first. In case there is a corresponding change to the testsuite rebase, push and create a MR first. We assume feature_123 is commited.

  1. git checkout master
  2. git merge --squash feature_123
  3. In case there is a corresponding teststuite: git add Testsuite
  4. git commit
  5. git push origin master:feature_123 --force or git push origin master:testing --force
  6. create merge request

amend - changing the last commit

Say you noticed a typo in you last commit message, or you forgot a file

  1. make the necessary changes, e.g. git add my-forgotten-file
  2. commit them with git commit --amend

SVN to GIT

The largest change is that one has a local copy of the repository, and can choose what to push to the remote. Commits are always local, and will only be shared upon git push. There is a good guide and a cheat-sheet for commands. Many commands are similar, the ones that differ are

checkout / clone

git clone URL # will clone the complete repository

update / pull

git pull --rebase # will fetch remote changes and rebase your local changes on top of them

to set the default for pull do

git config [--global] pull.rebase true

tag

Git tags are simply references to the repository state (SHA1) of any commit. Create and list them with

git tag -a name
git tag -l
git show <tagname>

branching

Branches are created locally and can be pushed to the remote

git branch # lists branches
git branch <branch-name> # creates branch 
git checkout <branch-name> # switches to branch
git checkout -b <branch-name> # as a one-liner

merge

Merge is local in git. Typically, merging branches is much better in git than svn. Make sure you are on one of the to-be-merged branches and merge the other one now:

git merge <branch-name>

You can also define aliases (see below) to use the svn commands you are used to.

Tips

Switch on colors

git config --global color.ui auto
git config --global color.diff auto
git config --global color.status auto
git config --global color.branch auto

Set mergetool to vimdiff

git config --global merge.tool vimdiff
git config --global merge.conflictstyle diff3
git config --global mergetool.prompt false

Possible mergetools are meld, opendiff, kdiff3, tkdiff, xxdiff, tortoisemerge, gvimdiff, diffuse, ecmerge, p4merge, araxis, vimdiff, emerge.

Show all branches

git branch -a

List information about repository (e.g. URL)

git remote show origin

Define aliases

git config --global alias.ci 'commit -v -a'
git config --global alias.st 'status'
git config --global alias.co 'checkout'
git config --global alias.info 'remote show origin'

To save password when using https you may modify the remote-url in .git/config to

[remote "http"]
         url = https://username:password@cfs-dev.mdmt.tuwien.ac.at/cfs/CFS.git
         fetch = +refs/heads/*:refs/remotes/http/*

however, this is not recommended -> use SSH.

You can define a shell prompt telling you which branch you are on:

  1. download git-prompt.sh and source it in your .bashrc
  2. Change your prompt to call __git_ps1, e.g. export PS1='[\u@\h \W$(__git_ps1 " (%s)")]\$ '

gitlab has the option ignore whitespace changes for the diff view. This can be helpful, if someone changed the indentation of a file.
Unfortunately gitlab does not support this for blame. However, you can still checkout the file and perform git blame -w FILE in a terminal.

Clone this wiki locally