Begin updating docs

JrGoodle · Jun 11, 2020 · dd000b7 · dd000b7
1 parent 18dba9a
commit dd000b7
Show file tree

Hide file tree

Showing 5 changed files with 468 additions and 52 deletions.
diff --git a/README.md b/README.md
@@ -14,20 +14,9 @@
 
 > **herding cats** - An idiom that refers to a frustrating attempt to control or organize a class of entities which are uncontrollable or chaotic
 
-Managing multiple repositories can be pretty frustrating. There are a number of existing options:
+Managing multiple repositories can be pretty frustrating. There are a number of existing options, the primary being git submodules and subtrees. Google's [repo](https://code.google.com/p/git-repo) tool takes a different approach, but is closely tied to Google's development workflow. `clowder` uses a similar approach as `repo`, but without the ties to Google's Android workflows. Detailed information about projects are specified in a `clowder.yml` (or `clowder.yaml`) file, but each repository is still essentially independent. Projects can track branches, or be tied to specific tags or commits. This file can be checked into its own repository so it can be versioned and shared across teams. The primary purpose of `clowder` is synchronization of multiple repositories, so normal development still takes place in individual repositories with the usual `git` commands
 
-- [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules)
-- [subtree merging](https://git-scm.com/book/en/v1/Git-Tools-Subtree-Merging)
-- [Google's repo tool](https://code.google.com/p/git-repo/)
-- [braid](https://github.com/cristibalan/braid)
-- [gr](https://github.com/mixu/gr)
-- [git-subrepo](https://github.com/ingydotnet/git-subrepo)
-
-All of these have their own approach, but many are based on submodules or subtrees. Submodules and subtrees create a tight coupling between repositories because of the way dependencies are stored. Much has been written about their drawbacks elsewhere. Google's `repo` tool takes a different approach, but is closely tied to Google's development workflow
-
-`clowder` uses a similar approach as `repo`, but using yaml instead of xml for the configuration file. URL information and relative project locations on disk are specified in a `clowder.yml` or `clowder.yaml` file. This file is checked into its own repository. The use of a separate file to track projects allows for detailed information about the dependencies between them to be stored, but each repository is still essentially independent. Projects can track branches, or be tied to specific tags or commits
-
-The primary purpose of `clowder` is synchronization of multiple repositories, so normal development still takes place in individual repositories with the usual `git` commands
+TODO: Why clowder?
 
 ## Getting Started
 
@@ -36,68 +25,115 @@ The primary purpose of `clowder` is synchronization of multiple repositories, so
 - [git](https://git-scm.com)
 - [Python 3](https://www.python.org/downloads/)
 
-#### macOS
+### Installation
 
-macOS comes with `git` preinstalled. To install Python 3 with [Homebrew](https://brew.sh)
+To install or upgrade `clowder` from PyPI
 
 ```bash
-brew install python
+sudo pip3 install clowder-repo --upgrade
 ```
 
-#### Ubuntu 16.04
+To install the latest pre-release version
 
 ```bash
-sudo apt install git
-sudo apt install python3-pip
+sudo pip3 install clowder-repo --force-reinstall --pre
 ```
 
-#### Windows
+## The clowder.yml file
+
+```yaml
+name: my-first-clowder
+
+defaults:
+  protocol: ssh
+
+sources:
+  - name: github
+    url: github.com
+
+projects:
+  - name: jrgoodle/mu
+  - name: jrgoodle/duke
+  - name: jrgoodle/kit
+    groups: [black-cats]
+  - name: jrgoodle/kishka
+    groups: [black-cats]
+  - name: jrgoodle/june
+    groups: [black-cats, notdefault]
+  - name: jrgoodle/sasha
+    groups: [black-cats, notdefault]
+```
 
-When using with [Cygwin](https://cygwin.com/install.html) the following dependencies should be installed
+### Breakdown
 
-- `git`
-- `python3-pip`
-- `python3`
+```yaml
+name: my-first-clowder
+```
 
-When using in the Windows CLI, `git` and a compatible Python version should be installed (see [appveyor.yml](https://github.com/JrGoodle/clowder/blob/docs/appveyor.yml) for supported Python versions on Windows)
+The `name` is simply a descriptive identifier. It must be a string that doesn't contain any spaces
 
-### Installation
+```yaml
+defaults:
+  protocol: ssh
+```
 
-To install or upgrade `clowder` from PyPI
+The `defaults` section requires the `protocol` to use for cloning repositories. The default `remote` is assumed to be `origin` and the default `branch` is assumed to to be `master`
 
-```bash
-sudo pip3 install clowder-repo --upgrade
+```yaml
+sources:
+  - name: github
+    url: github.com
 ```
 
-To install the latest pre-release version
-
-```bash
-sudo pip3 install clowder-repo --force-reinstall --pre
+The `sources` section contains all the places to clone repositories from.
+
+```yaml
+projects:
+  - name: jrgoodle/mu
+  - name: jrgoodle/duke
+  - name: jrgoodle/kit
+    groups: [black-cats]
+  - name: jrgoodle/kishka
+    groups: [black-cats]
+  - name: jrgoodle/june
+    groups: [black-cats, notdefault]
+  - name: jrgoodle/sasha
+    groups: [black-cats, notdefault]
 ```
 
-## Usage
+A project requires at minimum the `name` of the repository. This is combined with the `protocol` and `source` `url` to form the full url for cloning the repository, taking the form of  `git@${source.url}:${project.name}.git` or `https://${source.url}/${project.name}.git` depending on the protocol specified. If no `path` is specified, the last component of the `name` is used for the directory the project is cloned to. Projects can specify custom `groups` to run commands for only certain projects. By default, all projects are added to the `all` group, and a group of their `name` and `path`. If `notdefault` is present then the project will not be included in commands unless another group argument is given that it belongs to.
+
+There's much more cusomization possible with `clowder`. For some more complex examples see:
+
+[Cats clowder.yml](docs/clowder-yml-cats.md)
 
-This example is based on the [Swift projects](https://github.com/apple/swift) (see the full [Swift projects clowder.yaml](https://github.com/JrGoodle/swift-clowder/blob/master/clowder.yaml))
+[Forks clowder.yml](docs/clowder-yml-forks.md)
 
-First reate a directory to contain all the Swift projects
+[clowder.yml Syntax](docs/clowder-yml-syntax.md)
+
+## Usage
+
+First create a directory where all the prrojects will be cloned
 
 ```bash
-mkdir swift-source
-cd swift-source
+mkdir cats
+cd cats
 ```
 
 ### `clowder init`
 
+It's possible to just create a local `clowder.yml` file, but it's recommended to check the file into the root of a dedicated repository. This exammple uses an [existing repository](https://github.com/JrGoodle/clowder-examples) containing a `clowder.yml` file
+
 ```bash
-clowder init git@github.com:jrgoodle/swift-clowder
+clowder init git@github.com:JrGoodle/clowder-examples.git
 ```
 
 ![clowder init](docs/README/clowder-init.png)
 
 The `clowder init` command does the following
 
-- Clones the [Swift clowder repo](https://github.com/JrGoodle/swift-clowder) in the `swift-source/.clowder` directory
-- Creates a symlink pointing to the primary `clowder.yaml` file in the repository
+- Clones the [examples clowder repo](https://github.com/JrGoodle/clowder-examples) in the `cats/.clowder` directory
+- Creates a symlink in the `cats` directory: `clowder.yaml` -> `.clowder/clowder.yml`
 
 ### `clowder herd`
 
@@ -112,19 +148,17 @@ clowder herd
 - If any projects don't have a clean git status then `clowder` exits
 - Projects are cloned if they don't currently exist
 - Each project fetches the latest changes
-- If the current git ref checked out doesn't match the `clowder.yaml` configuration, the correct ref will be checked out
+- If the current git ref checked out doesn't match the `clowder.yml` configuration, the correct ref will be checked out
 - The latest changes are pulled for branches. For commits and tags, the commits are checked out into a detached `HEAD` state
 
-It's also possible to run `clowder herd` in parallel
-
-![clowder herd parallel](docs/README/clowder-herd-parallel.png)
-
 ### `clowder status`
 
 ```bash
 clowder status
 ```
 
+`clowder status` prints the current state of all projects
+
 ![clowder status](docs/README/clowder-status.png)
 
 ## Further Information
@@ -133,17 +167,19 @@ clowder status
 
 ```bash
 clowder branch # Print all local branches
-clowder checkout my_branch # Checkout my_branch in projects
+clowder checkout 'my_branch' # Checkout 'my_branch' in projects
 clowder clean # Discard any changes in projects
+clowder config get # EXPERIMENTAL: Get config values
+clowder config set projects 'my_goup' # EXPERIMENTAL: Set config values
+clowder config clear projects # EXPERIMENTAL: Clear config values
 clowder diff # Print git diff for all projects
 clowder forall -c 'git status' # Run command in all project directories
-clowder herd -b my_branch # Herd a specified branch if it exists, otherwise use default ref
-clowder link 0.1 # Set clowder.yaml symlink to a previously saved version
+clowder link 'v0.1' # Set clowder.yaml symlink to a previously saved version
 clowder repo run 'git status' # Run command in .clowder directory
-clowder save 0.1 # Save a version of clowder.yaml with current commit sha's
-clowder start my_feature # Create new branch 'my_feature' for all projects
+clowder save 'v0.1' # Save a version of clowder.yaml with current commit sha's
+clowder start 'my_feature' # Create new branch 'my_feature' for all projects
 clowder stash # Stash changes in all projects
-clowder prune stale_branch # Prune branch 'stale_branch' for all projects
+clowder prune 'stale_branch' # Prune branch 'stale_branch' for all projects
 ```
 
 For more information, see [the docs](http://clowder.readthedocs.io/en/latest/)

diff --git a/docs/clowder-yml-cats.md b/docs/clowder-yml-cats.md
@@ -0,0 +1,111 @@
+# Cats `clowder.yml` example
+
+```yaml
+name: cats-clowder
+
+defaults:
+  protocol: ssh
+  branch: tracking_branch
+  git:
+    config:
+      alias.cat: '!echo "😸"'
+
+sources:
+  - name: github
+    url: github.com
+
+projects:
+  - name: jrgoodle/mu
+    branch: groom
+    git:
+      lfs: true
+      recursive: true
+  - name: jrgoodle/duke
+    tag: v0.01
+  - name: jrgoodle/kit
+    path: black-cats/kit
+    commit: f2e20031ddce5cb097105f4d8ccbc77f4ac20709
+  - name: jrgoodle/kishka
+    path: black-cats/kishka
+    git:
+      config:
+        alias.cat: '!echo "😸😸😸"'
+  - name: jrgoodle/june
+    path: black-cats/june
+    git:
+      config:
+        alias.cat: null
+  - name: jrgoodle/sasha
+    path: black-cats/sasha
+    remote: catnip
+```
+
+## Breakdown
+
+```yaml
+defaults:
+  protocol: ssh
+  branch: tracking_branch
+  git:
+    config:
+      alias.cat: '!echo "😸"'
+```
+
+This example specifies a default `branch` of `trracking_branch` that all projects will inherit. It's also possible to specify a default `tag` or `commit`. The `git`section can contain custom git config entries that will be installed for all projects.
+
+```yaml
+- name: jrgoodle/mu
+  branch: groom
+  git:
+    lfs: true
+    recursive: true
+```
+
+This project will track the `groom` `branch`. The `git` configuration enables git lfs and submodules. Running `clowder herd` will install git lfs hooks, pull lfs files, and clone submodules recursively.
+
+```yaml
+- name: jrgoodle/duke
+  tag: v0.01
+```
+
+This project will check out the repository to the commit the `v0.01` tag points to.
+
+```yaml
+- name: jrgoodle/kit
+  path: black-cats/kit
+  commit: f2e20031ddce5cb097105f4d8ccbc77f4ac20709
+```
+
+This project will check out the repository to the commit specified by the full sha-1. The `path` the repository will be cloned at is `black-cats/kit`
+
+```yaml
+- name: jrgoodle/kishka
+  path: black-cats/kishka
+  git:
+    config:
+      alias.cat: '!echo "😸😸😸"'
+```
+
+When specified at the project level, the same git config values override any of the same ones in the `defaults`, in this case an alias for a `git cat` command that prints 😸.
+
+```yaml
+- name: jrgoodle/june
+  path: black-cats/june
+  git:
+    config:
+      alias.cat: null
+```
+
+To unset a git config entry in `defaults`, set the value to `null`
+
+```yaml
+- name: jrgoodle/sasha
+  path: black-cats/sasha
+  remote: catnip
+```
+
+Adding a `remote` entry will create a remote named `catnip` instead of `origin`
+
+## Usage
+
+TODO