Git support for LambdaCD
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
dev-resources make test output less noisy by default Jan 3, 2016
doc start guide on how to migrate from lambdacds built-in git support Mar 28, 2016
example/lambdacd_git/example Remove use of deprecated namespaces in examples so they work in Lambd… Mar 18, 2018
scripts Fix permissions of testrepo-ssh key Mar 18, 2018
src/lambdacd_git Log exception when unable to add custom identity file Feb 25, 2018
test/lambdacd_git Make end-to-end-test to use identity-file parameter Feb 25, 2018
.gitignore skeleton Dec 19, 2015
.travis.yml Use GitLab credentials with read and write permissions for tests Jun 16, 2018
CHANGELOG.md Bump dependencies to support clojure 1.9 (flosell/lambdacd#178) Mar 18, 2018
CONTRIBUTING.md Add contribution guide Jul 3, 2016
LICENSE
README.md Clarify git notifications in README Jun 11, 2018
go Update GitLab test token, remove hardcoded value Jun 11, 2018
project.clj Version 0.4.2-SNAPSHOT Mar 18, 2018

README.md

Git support for LambdaCD

Provides Git support for LambdaCD. Will replace the lambdacd.steps.git namespace in the the lambdacd-git library.

Status

Build Status

Clojars Project

Usage

;; project.clj
:dependencies [[lambdacd-git "<most recent version>"]]
;; import:
(:require [lambdacd-git.core :as lambdacd-git])

Complete Example

You'll find a complete example here: example/simple_pipeline.clj

Waiting for a commit

(defn wait-for-commit-on-master [args ctx]
  (lambdacd-git/wait-for-git ctx "git@github.com:flosell/testrepo"
                             ; how long to wait when polling. optional, defaults to 10000
                             :ms-between-polls 1000
                             ; which refs to react to. optional, defaults to refs/heads/master
                             :ref "refs/heads/master"))

; you can also pass in a regex:
(defn wait-for-commit-on-feature-branch [args ctx]
  (lambdacd-git/wait-for-git ctx "git@github.com:flosell/testrepo"
                             :ref #"refs/heads/feature-.*"))

; you can also pass in a function
(defn wait-for-commit-on-any-tag [args ctx]
  (lambdacd-git/wait-for-git ctx "git@github.com:flosell/testrepo"
                             :ref (fn [ref] (.startsWith ref "refs/tags/"))))

Using Web- or Post-Commit Hooks instead of Polling

wait-for-git can be notified about changes in a repository through an HTTP endpoint. To use this, you need to add it to your existing ring-handlers:

(ring-server/serve (routes
                         (ui/ui-for pipeline)
                         (core/notifications-for pipeline)) ; <-- THIS
                       {:open-browser? false
                        :port          8082})

This adds an HTTP endpoint that can receive POST requests on <host>/notify-git?remote=<remote>, e.g. http://localhost:8082/notify-git?remote=git@github.com:flosell/testrepo

Notice that this doesn't necessarily trigger a new build. Similar to how Jenkins works, a notification just forces a check for changes.

Cloning a Repository

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (:cwd args)))

(def pipeline-structure
  `(; ...
    (with-workspace
      clone
      do-something)))

; Works well with wait-for-git: 
; If no revision is given (e.g. because of manual trigger), clone falls back to the head of the master branch

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo (:revision args) (:cwd args)))

(def pipeline-structure
  `((either
      wait-for-manual-trigger
      wait-for-git)
     (with-workspace
       clone

       do-something)))

Get details on commits since last build

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (:cwd args)))

(def pipeline-structure
  `(wait-for-git
    (with-workspace
      clone
      lambdacd-git/list-changes
      do-something)))

Working with more than one repository

You can have clone steps that clone into different subdirectories:

(defn clone-foo [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (str (:cwd args) "/" "foo")))
(defn clone-bar [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (str (:cwd args) "/" "bar")))

(def pipeline-structure
  `(; ... 
    (with-workspace
      clone-foo
      clone-bar
      do-something)))

If you want to use this in combination with wait-for-git, you need to detect which commit to use. For details, see example/multi_repo_pipeline.clj

Tagging versions

You can tag any revision:

(defn deploy-to-live [args ctx]
  ;do deploy
  (let [cwd (:cwd args)]
    (lambdacd-git/tag-version ctx cwd repo "HEAD" (str "live-" version))))

(def pipeline-structure
  `(; ...
    (with-workspace
      ;...
      deploy-to-live
      do-something)))

Configuration (versions >= 0.3.0)

Certain values can be configured using LambdaCDs config-map. The following example shows the default:

(let [config  {:git {:timeout              20                               ; the timeout for remote operations in seconds
                     :credentials-provider (CredentialsProvider/getDefault) ; the credentials-provider to use for HTTPS clones (e.g. UsernamePasswordCredentialsProvider)
                     :ssh {:use-agent                true                         ; whether to use an SSH agent
                           :known-hosts-files        ["~/.ssh/known_hosts" 
                                                      "/etc/ssh/ssh_known_hosts"] ; which known-hosts files to use for SSH connections 
                           :identity-file            nil                          ; override the normal SSH behavior and explicitly specify a key to use
                           :strict-host-key-checking nil}}}]                      ; override the normal SSH behavior and explicitly set the StrictHostKeyChecking setting. Off by default, can be set to yes,no or ask
  (lambdacd/assemble-pipeline pipeline-structure config))

The same parameters can also be used as parameters to git operations directly, e.g. if different repositories need different credentials:

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (:cwd args) 
                      :credentials-provider (UsernamePasswordCredentialsProvider. (System/getenv "LAMBDACD_GIT_TESTREPO_USERNAME")
                                                                                  (System/getenv "LAMBDACD_GIT_TESTREPO_PASSWORD"))))

Configuration (versions < 0.3.0)

SSH Configuration

LambdaCD-Git honors the default SSH Config files from ~/.ssh/config(/etc/ssh/ssh_config currently not supported (see #23)). Use this to configure things like StrictHostKeyChecking or the IdentityFile. Alternatively, some options can be configured using ssh-init.

Authentication

Git over SSH

LambdaCD Git automatically picks up SSH-Keys in the default locations, e.g. ~/.ssh/id_rsa. SSH Agents are also supported.

Git over HTTPS

Authentication for HTTPS is supported using an instance of the JGit CredentialsProvider.

Import e.g. UsernamePasswordCredentialsProvider into your namespace:

(ns <your-pipeline-name>.core
  ;...
  (:import org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider)
  ;...
  )

Add an instance to the LambdaCD config:

(let [config {:home-dir "/some/path"
              :git {:credentials-provider (UsernamePasswordCredentialsProvider. "some-username" "some-password")}}]
              ; ... 
              )

Customize SSH Client

Some features of lambdacd-git (ssh-agent support, extended known_hosts support) require customizations to JGits singleton SSH session factory. Call init-ssh! once, e.g. in your -main function:

(defn -main [& args]
  ; ...
  (lambdacd-git/init-ssh!)
  ; ...
  )

Development

Call ./go

License

Copyright © 2015 Florian Sellmayr

Distributed under the Apache License 2.0