Beginners introduction to Git and GitHub
- Date and time: 3rd October 2018, 10:00 - 12:00
- Location: Drosophila Connectomics Group - Jefferis lab, Department of Zoology, University of Cambridge, UK
- Please bring your laptop or mobile device if you'd like to follow the practical part
Morning session (Sergio and Mark) https://tinyurl.com/git-intro-zoology:
- Background and motivation
- What is version control? What is Git? What is GitHub?
- How can you use Git and GitHub? How can they be useful for you?
- Practical session: working with Git and GitHub
Afternoon session (Mark and Anne) https://tinyurl.com/git-course: markdown, pages and wikis; creating good README files; issue tracking; sofware licenses
Background and motivation
When repeating / reviewing previous work, researchers greatly benefit from having access to detailed documention of the methods used.
Keeping track of the different versions of your project is one more way of being more reproducible.
Some of the manual approaches to version control have clear limitations:
- The scientific community is beginning to consider the value of peer reviewing computer code - see Nature Methods August 2018 editorial Easing the burden of code review:
An increasing share of modern research relies on analytical code and software. In turn, a good deal of irreproducible research can be attributed to computational tools that are difficult to decipher, use or recreate. Through the concerted efforts of computational researchers and stricter guidelines from publishers, the culture of scientific software is now more open and geared toward dissemination than ever [...]
- GitHub is becoming the go-to site when it comes to releasing / sharing the code associated to a manuscript or the scripts developed within a project.
What is version control? What is Git? What is GitHub?
Version control is the management of changes (a.k.a. revisions) to any types of information
- Simple versioning: adding v1.0, v1.1, v1.2, v2.0 ... to file names
- Basic tools: Google Drive, Dropbox ...
- Advanced tools: Git
The first version control systems were created by groups writing software and code. Fortunately they can now be used not only for computer code but for any type of files
(adapted from http://lhzuigao.com/309note.html)
Advantages of distributed (right) over centralised (left) version control systems involve:
- If the central repository (server) crashes, it could be recovered / backed up from any of the local repositories created e.g. by the researcher, collaborator or group leader.
- Each person can make changes to their local repositories offline. Then integrate their individual changes in the central repository (server) when connected online.
Git is a distributed version control system to keep track and compare the history of changes made to your scripts and files. It allows groups of people to work on the same documents at the same time, and without stepping on each other's toes. It was created by Linus Torvalds in 2005 for the development of the Linux project. It is free and open source and helps you with:
- Creating repositories to host your projects using the command-line
- Tracking changes in the files and folders within your repositories
GitHub is a platform to share and showcase your work online with collaborators and the wider audience. A tool to help you build projects that are collaborative, well documented, and version-controlled. It provides you with:
- A place to host and backup your repositories online
- A nice web interface to your repositories
- A strategy to collaborate with colleagues
Versions in Git and GitHub are identified by a revision number, e.g. 60363b1, also known as commit. Each revision is associated with a timestamp and the person making the change. Revisions can be compared, restored, and with some types of files, merged.
How can you use Git and GitHub? How can they be useful for you?
The interfaces to Git and GitHub are:
- Via the command-line using git
- Directly online
- Github Desktop (available for Mac and Windows)
For this workshop, we will use Git commands and GitHub's online interface.
To host and share your research outputs and software
To create websites using GitHub pages
- Personal research websites, e.g. Mike Love site
- Courses and activities, e.g.
To write your PhD thesis, e.g. A reasoning framework for C4 photosynthesis research based on high-throughput analysis
In the context of our research group
Communication is key as most projects have both experimental and computational leaders
Building from the classical ways of sharing - conversations/meetings, email, Dropbox, shared folders ... we want to build an environment where:
- Computational colleagues can share code, figures and tables. Review others work and get credit from their collaborative work
- Experimental colleagues can follow computational developments, access results and learn methods of data analysis
And ideally avoiding situations like ...
- Parhaps a happier lifetime for a research project:
Public (free) and private repositories
If you want to start creating repositories in GitHub, your first need to open an account:
- Public repositories are free, and can be browsed and downloaded by anyone
- Private repositories have associated costs - see pricing of plans. The developer plan costs $7/month but it is free if you are a student or an academic
GitHub uses Markdown for text edition, a language with plain text formatting syntax (bold, italics, checkboxes, lists, etc.), to render pages online (like HTML but easier). You can use this syntax in text files (.md), commit messages, issues, blog posts, and more.
Markdown is important because GitHub automatically renders anything written in Markdown. This can be specific files (eg: README), or your comments and issues.
Some examples of Markdown syntax are available here.
Practical session: working with Git and GitHub
We have four possible tutorials:
- Create a GitHub account (+)
- Create your first repository (+)
- Explore your first repository and GitHub account (++)
- Making changes using Git in the command-line (+++)
Create a GitHub account
- Go to https://github.com
- Fill in your Username, Email and Password. Then click on the green button "Sign up for GitHub".
- Choose your personal plan page. Select "Free plan" and then click on "Continue".
- Tailor your experience page. Choose the boxes that apply to you and click on "Submit". Otherwise, just go to "skip this step".
- You have created a GitHub account!
Create your first repository
- If you are not already signed in, sign in to GitHub using the Username/Email and Password created before.
- Click on the top-right "avatar icon" and select "Your profile". Have a quick browse through your page.
- Click on the top-right "+" icon and select "New repository". Verify your email address. You should have just received an email from GitHub in the address provided before. Find this email and click on "Verify email address".
- Create a new repository page. Fill in a "Repository name", e.g. "my_first_repository" or "my_analysis_script". Write a short description of your repository e.g. "This is a test repository". For now choose "Public" and select the box to initialize this repository with a README. Finally, click on "Create repository".
- You created your first repository!
Explore your first repository and GitHub account
Your first repository
- Click on README.md and go to the right pencil "Edit this file". Type anything to change the file, e.g. "GitHub is fun!".
- Scroll down. Introduce a commit change message, e.g. "My first update", and select the radio button "Commit directly to the master branch". Then click on "Commit changes". Voilá!
- To view your history of commits for README.md, click on README.md and then on the "History" button on the right.
- Alternatively, to view your history of commits for your first repository, click on the name of your repository and select the tab depicting a small clock and the number of commits next to it.
Bonus points (5 min):
- Try to create a new file
- In your new repository, have a look at the "Settings" tab, explore "Collaborators" and try to add the person sitting next to you.
Your GitHub account
- Click on your top-right "avatar" icon and select "Settings".
- Explore the tabs "Profile", "Account" and "Emails".
Repository: it can be thought of as a project folder. A repository contains all of the project files, issues, wikis and more. It also stores the history and versions of each file.
Commit: equivalent to saving your changes to a file. When you commit you usually include a brief description of the changes you made so you can identify versions later if you want to undo a change.
Branch: an identical copy of a project at a particular point in time kept separate from the 'master' branch (primary copy). This keeps your code in the 'master' branch safe while you make changes and experiment with code on the new branch. You can merge your new branch back into the 'master' branch when you want to publish your changes.
Master: the default branch in your repository.
Collaborator: someone with read and write privileges to a repository as approved by the repository owner.
Making changes using Git in the command-line
Check if Git is already installed in your computer, otherwise install it
(If in Mac), go to Finder -> Applications -> Utilities -> Terminal and type
- If you get as output something like
git version 2.5.4 (Apple Git-61), then Git is already installed -> Jump to the next section.
- If you get something around
git: command not found, keep reading.
- If you get as output something like
To install Git in Mac, follow one of the next strategies:
- When running one of the following commands
xcode-select --installyou may be offered to install developer command line tools. Accept the offer and follow with "Install".
- Go to https://git-scm.com/downloads and download git. Double click on the downloaded executable and follow instructions.
- If you have
homebrewinstalled, type the following in the Terminal:
brew install git.
- When running one of the following commands
Tell git who you are (your GitHub username) and what your email address is
cd ~/Desktop git config --global user.name "semacu" git config --global user.email "firstname.lastname@example.org"
Remember to change "semacu" and "email@example.com" to the username and email you used when creating the GitHub account above.
git config --list
Clone the repository created before
git clone https://github.com/semacu/my_first_repository.git cd my_first_repository ls -lh
Your first repository created using GitHub (my_first_repository) is now a local repository located in your Desktop folder. Remember what we discussed earlier about Git being a distributed version control system.
Tell git what's your remote repository url to pull and push commits
cd ~/Desktop/my_first_repository git remote set-url origin https://firstname.lastname@example.org/semacu/my_first_repository.git
git remote -v
Make a change to the
README.md file using your favourite text editor
- In your Desktop, use Finder to go to the cloned folder and open
README.mdwith your favourite text editor, e.g. TextEdit.
README.md, e.g. add a new line "This is my second line of script" and save changes.
- Now, go back to the Terminal and check how changes are tracked by Git:
cd ~/Desktop/my_first_repository git status
The status of
README.md is modified but the changes are not staged (red).
Stage and commit the change
git add README.md git status
The status of
README.md is modified and now the changes are
staged (green) and ready to commit.
git commit -a -m "My second update" git status
Push changes to your online GitHub repository
git push origin master
Now check that your change to
README.md made to your online GitHub repository.
Bonus points (5 min):
- Make another change to
README.mdusing the online GitHub repository and pull the change to your local repository (Hint: use
Clone: a copy of an online repository on your local computer so you can make edits on your own personal copy without having to be online. You can sync changes between your clone and the remote copy (GitHub) when you are online.
Remote: a version of your project repository that is hosted on the Internet or network somewhere (e.g. copy of your project on GitHub vs. on your local computer).
Stage and commit:
Push: sends the recent commit history from your local repository up to GitHub.
Pull: grabs any changes from the remote GitHub repository and merges them into your local repository.
- Create an issue
- Create a new branch, open a pull request and merge the newly created branch with the master branch
- Fork a repository from another user e.g. https://github.com/githubtraining/hellogitworld, make some changes to the
README.txtand create a pull request
A basic overview to the basic functionality of Git and GitHub
Next steps for computational reproducibility, going back to the Nature Methods August 2018 editorial Easing the burden of code review:
[...] Yet, even in the era of Git repositories, peer reviewing code can be frustrating and time consuming [...] Computational tools are complex objects that depend on many components to run. Dependencies include the operating system, programming language, external code libraries, configuration settings and run parameters. Reproducing these conditions is made even harder by the fact that components typically exist in multiple versions. Many come with their own prerequisites, creating a maddening rabbit hole of dependencies on dependencies [...]
In other words, future steps will be to be able to execute code directly online (cloud). Two new resources are beginning to make a difference in this area - check them out
- Code Ocean: Nature Methods, Nature Biotechnology and Nature Machine Intelligence have launched a trial to facilitate the peer review of computational methods and to improve their reproducibility
Many Thanks for your attention! Enjoy Git and GitHub!
Feedback: please complete the following short survey
References and materials
- A Friendly Introduction to GitHub
- Software Carpentry: Version Control with Git
- Resources to learn Git
- GitHub On Demand Training
- A quick introduction to Git and GitHub
- GitHub Help
- Nature Methods 2018 editorial, Easing the burden of code review
- Perkel 2018:
- Silver 2018 Microsoft’s purchase of GitHub leaves some scientists uneasy
- Russell et al. 2018 A large-scale analysis of bioinformatics code on GitHub
- Perez-Riverol et al. 2016 Ten Simple Rules for Taking Advantage of Git and GitHub
- Perkel 2016 Democratic databases: science on GitHub
- Markowetz 2015 Five selfish reasons to work reproducibly