Home

Welcome to an short introduction to the gitosis easy conf gem.

Motivation

Gitosis is a tool for managing the access rights to a bunch of git repos. The configuration is simple and easy to understand but can get very long and complex for lots of repos. Especially if there a number of groups with different access rights that may or may not overlap for specific repos.

It then becomes almost impossible to maintain an overview if the development process follows the Github fork and pull request style. Using the fork-pull-merge model means that each developer has their own copy of the main repo and places a pull-request to the main repo once they have something to merge. The advantage is that anything merged into the master repo will have been reviewed by at least two pairs of eyes and generally more since the pull-request is public ('public' for the internal project) and the entire development team can review it. Also a developer is free to experiment on their repo without risk to the master repo.

In addition, the access rights for the deploy users need to be set on all repos (deployment from a developer repo should be possible since you might want to deploy to a staging system using a specific version of the application). A deploy user has read-only rights to all repos that are to be deployed to the servers. (There might be an extra repo that contains passwords and access tokens that are linked into the deployed application.)

This is where gitosis easy conf comes in. It provides a DSL to allow gitosis configuration in a more precise and concise way. It's a kind of scripting language for gitosis.conf and gitosis.conf is the bytecode that is generated once the gitosis-easy-conf configuration is compiled ;)

Why so many repos?

In most projects, you'll end with a certain amount of separation of concern: you might have libraries that are used across your project, you might have a separate repo for passwords (database + access tokens), you might have a chef repo for configuring your servers, and then you'll have your application codebase. And of course, you'll have a gitosis repo.

In addition, you might create a repo for hosting the codebase of your redmine installation. Of course, you might do the same with your continous integration server ... etc.

Of course, this can be covered by a hosted solution but if you can't afford that or you don't wish to host your intellectual property on some remote host, then you'll end up setting up your infrastructure from scratch. (As an aside, using chef and bare-metal servers that all have the same distro, this does become more a matter of rinse-and-repeat than hours of pain and hair pulling!)

Also, it might happen that you have github projects that you integrate into your project but you might like to make changes that are part of the core business and can't be open-sourced. (As evil as this might sound, you still need to earn a buck as a developer.) In that case, it's great having the ability to create a new git repo in an instant and manage that repo as part of your entire project. And you can open-source stuff by pushing to the original github repo while maintaining a branch as internal which is pushed to the internal git host.

Usage

The following is the scenario where we have two developers and one repo (overkill for the usage of gitosis easy conf) where dev.one can read+write to the main repo, dev.two has a fork of the original repo and can read the the original repo.

Gitosis.setup("gitosis.concise.conf") do
  config do
    # define the naming convention for repos that are fork repos
    # The proc is passed the name of the forker and the name 
    # of the repo being forked
    fork_naming_convention do |repo,forker|
      "#{repo}_been_forked_by_#{forker}"
    end
  end

  # list of all forkers. Here we give each forker a name and their public
  # key --> assumption being that in the keydir for gitosis there is a 
  # named dev.one.pub (for example)
  forkers do
    dev1 "dev.one"
    dev2 "dev.two"
  end

  # here we define a single repo that is writable from forker one, forker dev2 
  # has a fork repository and the dev2 can read the original repoistory (and write
  # to their fork repo).
  repositories do
    repo_name :writable => Forker[:dev1], :forks => :dev2, :readable => Forker[:dev2]
  end
end

the corresponding gitosis.conf is then:

[gitosis]

[group repo_name.writable]
members = dev.one
writable = repo_name

[group repo_name.readonly]
members = dev.two
readonly = repo_name

[group fork.repo_name.dev2.writable]
members = dev.two
writable = repo_name_been_forked_by_dev2

[group fork.repo_name.dev2.readonly]
members = dev.two
readonly = repo_name_been_forked_by_dev2

The gitosis.conf is fully denormalized which is ok since you shouldn't be editing it directly. The gitosis easy conf above should be placed in a Rakefile in the gitosis top-level repo. A task create then creates the gitosis.conf.

With groups

Groups provide for more precise rights management. In this case, we have four developers and three main rails applications. We also have two deploy users for our servers. Two developers are the gatekeepers to the master repositories. We don't have any common gems or shared code, just three application codebases.

Intention is that the deploy users can read the application codebase (to deploy it), the developers 3 and 4 can read the original repos (to merge to their forks) and developers 1 and 2 can read + write to master but also have their own fork repos. All developers should work in the fork repos, even developers 1 and 2. (A CI that allows for all developers to test their fork repos is a nice-to-have!)

Each developer should maintain their master branch on their fork with the master/master branch.

Gitosis.config do
  fork_naming_convention do |repo, forker|
    "#{forker}_#{repo}"
  end
  filename "gitosis.readme.conf"
end

Gitosis.forkers do
  developer1 'dev.one'
  developer2 'dev.two'
  developer3 'dev.three'
  developer4 'dev.four'
end

Gitosis.groups do
  admins         Forker[:developer1], Forker[:developer2]
  developers     Forker[:developer3], Forker[:developer4]
  deployers      'app1.deploy', 'app2.deploy'
  all_developers :admins, :developers
end

Gitosis.repositories do
  # :forks => :all means that all forkers have a forked repo of the main repositories.
  with_base_configuration( :writable => :admins,
                           :readable => [:all_developers, :deployers],
                           :forks    => :all ) do
    rails_app_one
    rails_app_two
    rails_app_three
  end
end

The resulting gitosis.conf is quite long and impossible to manage (IMHO).

With project wide gem

This use case has six developers (cto, lead developer and 4 developers), 3 rails applications, 1 project wide gem. Also we have a chef repo for managing our servers and a db_config repository for storing database and API tokens (these are linked into the applications on deploy).

The use of a db_config repo is a design decision to store sensitive data in separate repo and not on the servers. This has the advantage that servers are easily setup, disadvantage is that sensitive data is easily gotten at and centralized. For this reason, write access to the db_config repo is limited to the committers and read access only to the deploy users (db_config is updated on each deploy of each application). Of course this can also be stored in the chef repo (if being used) however this then makes porting a chef repo to a new project difficult (or at least, a little less obvious).

Administrative repos (chef + gitosis-admin) are only writable by the admins (cto, lead dev and unix-admin) but readable for all. These repos aren't forked and committed to directly. The assumption being that any changes are limited and usually done by one person at a time. Depending on the exact project setup, it is probably desirable that the chef repo is only writable by the unix admin and not the developers.

Also, access to the deploy users (i.e. authorized_keys) is would also be limited to the committers, i.e. cto + lead dev are the only two that can deploy and commit to master. Once again, this is a decision that is up to each team but the intention is to separate concerns and recognition of the fact that not all developers want the responsibility of deploying an application at 3 am in the morning when the product manager suddenly decides that a certain feature just needs to go live...

Gitosis.setup("gitosis.complex.conf") do
  config do
    fork_naming_convention do |repo, forker|
      "#{forker}s_fork_of_#{repo}"
    end
  end

  forkers do
    dev_cto      'dev_cto.key'
    dev_lead_dev 'dev_lead_dev.key'
    dev_one      'dev_one.key'
    dev_two      'dev_two.key'
    dev_three    'dev_three.key'
    dev_four     'dev_four.key'
  end

  # roles are an alias for groups. In the definition of groups, symbols are assumed to be 
  # references to either forkers or other groups and strings are assumed to be the explicit
  # name of a public key in the keydir.
  roles do
    committers Forker[:dev_cto], Forker[:dev_lead_dev]
    admins 'unix.admin', :committers

    blog_developers Forker[:dev_four], Forker[:dev_three]
    www_developers  Forker[:dev_four], Forker[:dev_two]
    api_developers  Forker[:dev_one],  Forker[:dev_three], Forker[:dev_two]

    all_developers :blog_developers, :committers, :www_developers, :api_developers
    deployers 'app1.deploy', 'app2.deploy', 'app3.deploy'
  end

  repositories do
    # rails applications (blog, www and api) and common gem.
    with_base_configuration(:writable => :committers,
                            :readable => [:all_developers, :deployers]) do
      blog_app :forks => [:dev_four, :dev_three, :dev_lead_dev]
      www_app  :forks => [:dev_four, :dev_two, :dev_lead_dev]
      api_app  :forks => [:dev_one,  :dev_three, :dev_two, :dev_lead_dev]
      common_gem :writable => :all_developers
    end

    # chef and gitosis-admin. Note there are no forks these repos, only master.
    with_base_configuration(:writable => :admins,
                            :readable => :all_developers) do
      chef
      # name overrides the repo name, so instead of gitosis_admin, the repo is called gitosis-admin.
      gitosis_admin :name => 'gitosis-admin'
    end

    # production database configuration
    db_config :writable => :committers, :readable => :deployers
  end
end

The resuling gitosis.conf is a nightmare, but in a sense the result of being consequent with idea of separation of concern: setting up servers is something different than a application, an api application does need to be part of the web application, passwords should not be store in the application codebase, etc.

In combination with Redmine

Combining all these repos with Redmine is, initially, a time-consuming activity. However, it is easy enough to extend Redmine to allow project creation via API. So that creating a new gitosis repo also automagically creates a new redmine project. And since Redmine is nothing more than a rails application, we use capistrano to deploy our own redmine installation (just as we deploy all our rails applications) and manage our in-house modifications to redmine.

Our setup is such that each rails application has a sub-projects each of which is a developer fork of the rails application. This is easily done once the project for a fork is created in redmine. In addition, pull-requests are represented in redmine as a separate issue state. Usually an feature-issue is initial implemented, then given to another developer for review and finally pushed on a commit developer as a pull-request.

Branch naming convention

It is easy enough to reference commits across any repository in redmine (although this only works with projects that are sub-projects, not across projects) however this can also be time-consuming if a developer needs to do this explicitly.

This is where a git-hook that takes a branch name and includes the ref #XX automatically in the commit message comes in handy. So development on a feature can be done on a branch named issue_XXX_some_postfix and when committed, the commit message becomes:

some commit message

refs #XXX

On branch: issue_XXX_some_postfix

This then ensures the commit is associated with the issue and all commits related to the issue across all repositories are tracked.

Examples

All the above usage examples can be found on the examples directory of the gem.