index.org

Logging and Backing up your Work

This seminar took place on June 7, 2016 @ 1:30PM-4:30PM (UTC+2). The link to the video is below.

Table of Contents

• Practical Information
  • Webcast/Video
  • Persons in charge
  • Locations
• Practical Session Requirements
  • Gitting Started (courtesy of HackBerkeley)
  • Installing a GUI: SmartGit
• Git-labbook Demo
  • Installation of Git-labbook
  • Parallel Quicksort Study
  • Demo

Practical Information

Vincent Danjean, Arnaud Legrand and Luka Stanisic provided a (not so short) introduction to git, a distributed version control system, that will allow you to efficiently and reliably log and back up your work.

Webcast/Video

Here is the link to the video. If you have any trouble with the flashplayer, here is a direct link to the mpeg4 video.

Here are the slides.

To interact during the presentation (questions, comments), we used this pad that we will keep for the record.

Persons in charge

• Grenoble: Arnaud Legrand, Frédéric Desprez, Pierre Neyron
• Bordeaux: Stanisic, Emmanuel Agullo, Nicolas P. Rougier, Ludovic Courtès, …
• Rennes: Martin Quinson
• Besançon: Louis-Claude Canon

Locations

• Grenoble: Imag Amphitheater on Campus

Practical Session Requirements

Basic Git installation and your favorite GUI (unless you are really hardcore and prefer doing everything in command line).

Gitting Started (courtesy of HackBerkeley)

Operating System	Download/Install
OS X	Download from http://git-scm.com/download/mac
Windows	Download from http://git-scm.com/download/win - Two programs will be installed: Git BASH and Git GUI. Use Git BASH! Although the GUI can perform almost all the same functions as BASH, it’s great to get experience working from the command line. You can use unix/linux commands within BASH.
Linux	Ubuntu, Debian, and Mint users can run from their terminal: sudo apt-get install git. Fedora Users: sudo yum install git

Installing a GUI: SmartGit

There are many graphical interfaces for Git and we recommend using SmartGit, for the following reasons:

• It is nice and shiny ☺
• It has smart default configurations for Git commands
• It can contain multiple projects
• It works with Mercurial and SVN as well
• It is portable to any operating system (Linux, Windows, OS X)
• It is free (for non-commercial use)
• It is up-to-date, supporting advanced Git features
• It is well integrated with popular platforms (GitHub, BitBucket, etc.)

Unfortunately, SmartGit is not open source and it is written in Java, but there is no free lunch.

To install SmartGit, download it from the following page: http://www.syntevo.com/smartgit/download and follow the instructions. The first time you launch the application, in the first step, you will be asked to choose your license type. Be sure to change it for “Non-commercial use only” (unless you are working on a commercial code of course). For the rest of the installation, unless you know exactly what you are doing, we suggest to you to keep the default options.

Git-labbook Demo

Git-labbook is a git branching model adapted to experimental research. More information can be found on the project homepage.

Installation of Git-labbook

You can clone git-labbook from GitHub and install it using Makefiles.

git clone git@github.com:Git-labbook/git-labbook.git git-labbook
cd git-labbook

dzil build

make Makefile.PL
make

At this point, there are certain perl package requirements and the list is available on the project homepage. Please indicate if some packages are missing for you or if you have any problems installing the software.

Parallel Quicksort Study

To demonstate git-labbook commands, we are using an already performed reproducible research example RR_example. Commits of RR_example have been transformed into patches, that we simply apply in the following demo. This way, the main focus stays on Git branching system and git-labbook commands. However, we still encourage readers to explore the content of the RR_example study (especially LabBook.org) as it contains many useful tips about properly writing laboratory notebook.

Demo

Initialization

Cleanup and environment preparation.

set -ex

cd scenario.d 

mkdir -p repo
rm -rvf repo/* 
rm -rf repo/.??* 
rm -rvf repoMachine2/*
rm -rf repoMachine2/.??*

DATADIR="$(pwd)/data"
export PATH="$(pwd)/../blib/script:$(pwd)/../bin:/home/vdanjean/debian/mainteneur/org-merge-driver/upstream/org-merge-driver/build/src:$PATH"
export PERL5LIB="$(pwd)/../blib/lib:$(pwd)/../lib:$PATH"

Check that options are correctly connected to config variables.

git-labbook help

Initialize Git repository, add few commits.

mkdir -p repo
cd repo

git init
git am $DATADIR/000[1234]-*

Initialize Git-labbook. This creates the whole directory structure, laboratory notebook file and an initial commit.

git-labbook init

Adding some scripts into data branch.

git checkout data
git am $DATADIR/001[0]-*

Doing first experiments

Start first experimental campaign by creating a new experimental branch.

git checkout master
git-labbook xp start xp1 --branch xp1 --no-labbook-entry

Do some modifications, execute the experiments and add the measured data.

git am $DATADIR/000[56]-*

Continue doing some modifications inside the master branch. One may also realize that some code in the experimental branch is useful for the whole project, so it can be cherry-picked into the source branch.

git checkout master
git am $DATADIR/000[78]-*

git am $DATADIR/001[12345]-*

Go back to the experiment branch and improve laboratory notebook text.

git checkout xp1
git am $DATADIR/000[9]-*

When concluded that experiments are finished, merge the experimental branch into data branch.

git checkout xp1
git-labbook xp finish

Repeat a similar process for another experimental campaign.

git checkout master

git-labbook xp start xp2
git am $DATADIR/001[6]-*

git checkout data
git am $DATADIR/001[7]-*

git checkout xp/xp2
git-labbook xp finish

Additionally, when a certain important source code developments are performed, one can add a Git tag (typically for the code releases).

git checkout master
git am $DATADIR/001[9]-*
git tag stable0.9

Performing experiments on another machine

One might want to perform experiments on another machine. For that, first the project needs to be cloned. In this example, this is simply done in another folder of the same computer, but it can be done similarly from a different machine.

cd ..
git-labbook clone repo repoMachine2
cd repoMachine2

Start experiments on that machine.

git-labbook xp start xp3 -b xp3 --no-labbook-entry
git am $DATADIR/002[01234]-*

At some point, push the experiments to the remote repository.

git-labbook xp push

When the experimental campaign is finished, the experimental branch can be merged into the data branch.

cd ..
cd repo

git checkout xp3
git-labbook xp finish

At the end, the whole study is reproduced!

git checkout master
git am $DATADIR/002[6]-*

echo "SUCCESS REBUILD"