Merge pull request #468 from JrGoodle/refactor

Refactor git commands and add Python 3.8

.travis.yml

@@ -20,6 +20,9 @@ jobs:
   - env: TARGET="cats"
     python: 3.7
     script: clowder-test -c cats all
+  - env: TARGET="cats"
+    python: 3.8
+    script: clowder-test -c cats all
   - env: TARGET="parallel"
     python: 3.7
     script: clowder-test -c parallel

CHANGELOG.md

@@ -44,7 +44,7 @@
 
 ## 2.3.0
 
-- Add more options to `clowder clean` (`-f`, `-x`, -X`, `-d`, `-r`, `-a`)
+- Add more options to `clowder clean` (`-f`, `-x`, `-X`, `-d`, `-r`, `-a`)
 - More thorough cleaning logic
 - Add the ability to override project forks in imported `clowder.yaml` files
 - Add `clowder yaml` command to print `clowder.yaml` information

CONTRIBUTING.md

@@ -5,44 +5,44 @@
 Clone the repository:
 
 ```bash
-$ git clone git@github.com:JrGoodle/clowder.git
-$ cd clowder
+git clone git@github.com:JrGoodle/clowder.git
+cd clowder
 ```
 
 Add the directory you cloned the repo into to `PYTHONPATH` in your shell profile. For example, if cloned into `$HOME/clowder`
 
 ```bash
-export PYTHONPATH=$PYTHONPATH:$HOME/src/clowder
+export PYTHONPATH=$PYTHONPATH:$HOME/clowder/src/clowder
 ```
 
 ## Building
 
 Install `clowder` for local development
 
 ```bash
-$ script/update
+script/update
 ```
 
 Remove `clowder` and clean test directories
 
 ```bash
-$ script/clean all
+script/clean all
 ```
 
 ## Testing
 
 Install `clowder-test` command runner
 
 ```bash
-$ script/test
+script/test
 ```
 
 Run test scripts
 
 ```bash
-$ clowder-test swift all
-$ clowder-test cats herd
-$ clowder-test unittests
+clowder-test swift all
+clowder-test cats herd
+clowder-test unittests
 ```
 
 ## Pull requests

README.md

@@ -1,4 +1,5 @@
 # `clowder`
+
 [![Build Status](https://travis-ci.org/JrGoodle/clowder.svg)](https://travis-ci.org/JrGoodle/clowder)
 [![CircleCI](https://circleci.com/gh/JrGoodle/clowder.svg?style=shield)](https://circleci.com/gh/JrGoodle/clowder)
 [![Maintainability](https://api.codeclimate.com/v1/badges/56c92799de08f9ef9258/maintainability)](https://codeclimate.com/github/JrGoodle/clowder/maintainability)
@@ -11,7 +12,6 @@
 [![Documentation Status](https://readthedocs.org/projects/clowder/badge/?version=latest)](http://clowder.readthedocs.io)
 
 > **clowder** - A group of cats
-
 > **herding cats** - An idiom that refers to a frustrating attempt to control or organize a class of entities which are uncontrollable or chaotic
 
 Managing multiple repositories can be pretty frustrating. There are a number of existing options:
@@ -29,29 +29,29 @@ All of these have their own approach, but many are based on submodules or subtre
 
 The primary purpose of `clowder` is synchronization of multiple repositories, so normal development still takes place in individual repositories with the usual `git` commands
 
-# Getting Started
+## Getting Started
 
-## Requirements
+### Requirements
 
 - [git](https://git-scm.com)
 - [Python 3](https://www.python.org/downloads/)
 
-### macOS
+#### macOS
 
 macOS comes with `git` preinstalled. To install Python 3 with [Homebrew](https://brew.sh)
 
 ```bash
-$ brew install python
+brew install python
 ```
 
-### Ubuntu 16.04
+#### Ubuntu 16.04
 
 ```bash
-$ sudo apt install git
-$ sudo apt install python3-pip
+sudo apt install git
+sudo apt install python3-pip
 ```
 
-### Windows
+#### Windows
 
 When using with [Cygwin](https://cygwin.com/install.html) the following dependencies should be installed
 
@@ -61,46 +61,48 @@ When using with [Cygwin](https://cygwin.com/install.html) the following dependen
 
 When using in the Windows CLI, `git` and a compatible Python version should be installed (see [appveyor.yml](https://github.com/JrGoodle/clowder/blob/docs/appveyor.yml) for supported Python versions on Windows)
 
-## Installation
+### Installation
 
 To install or upgrade `clowder` from PyPI
 
 ```bash
-$ sudo pip3 install clowder-repo --upgrade
+sudo pip3 install clowder-repo --upgrade
 ```
 
-# Usage
+## Usage
 
 This example is based on the [Swift projects](https://github.com/apple/swift) (see the full [Swift projects clowder.yaml](https://github.com/JrGoodle/swift-clowder/blob/master/clowder.yaml))
 
 First reate a directory to contain all the Swift projects
 
 ```bash
-$ mkdir swift-source
-$ cd swift-source
+mkdir swift-source
+cd swift-source
 ```
 
-## `clowder init`
+### `clowder init`
 
 ```bash
-$ clowder init git@github.com:jrgoodle/swift-clowder
+clowder init git@github.com:jrgoodle/swift-clowder
 ```
 
 ![clowder init](docs/README/clowder-init.png)
 
 The `clowder init` command does the following
+
 - Clones the [Swift clowder repo](https://github.com/JrGoodle/swift-clowder) in the `swift-source/.clowder` directory
 - Creates a symlink pointing to the primary `clowder.yaml` file in the repository
 
-## `clowder herd`
+### `clowder herd`
 
 ```bash
-$ clowder herd
+clowder herd
 ```
 
 ![clowder herd](docs/README/clowder-herd.png)
 
 `clowder herd` updates the state of the projects. When the command is run, the following happens
+
 - If any projects don't have a clean git status then `clowder` exits
 - Projects are cloned if they don't currently exist
 - Each project fetches the latest changes
@@ -111,35 +113,35 @@ It's also possible to run `clowder herd` in parallel
 
 ![clowder herd parallel](docs/README/clowder-herd-parallel.png)
 
-## `clowder status`
+### `clowder status`
 
 ```bash
-$ clowder status
+clowder status
 ```
 
 ![clowder status](docs/README/clowder-status.png)
 
-# Further Information
+## Further Information
 
-## More commands
+### More commands
 
 ```bash
-$ clowder branch # Print all local branches
-$ clowder checkout my_branch # Checkout my_branch in projects
-$ clowder clean # Discard any changes in projects
-$ clowder diff # Print git diff for all projects
-$ clowder forall -c 'git status' # Run command in all project directories
-$ clowder herd -b my_branch # Herd a specified branch if it exists, otherwise use default ref
-$ clowder link -v 0.1 # Set clowder.yaml symlink to a previously saved version
-$ clowder repo run 'git status' # Run command in .clowder directory
-$ clowder save 0.1 # Save a version of clowder.yaml with current commit sha's
-$ clowder start my_feature # Create new branch 'my_feature' for all projects
-$ clowder stash # Stash changes in all projects
-$ clowder prune stale_branch # Prune branch 'stale_branch' for all projects
+clowder branch # Print all local branches
+clowder checkout my_branch # Checkout my_branch in projects
+clowder clean # Discard any changes in projects
+clowder diff # Print git diff for all projects
+clowder forall -c 'git status' # Run command in all project directories
+clowder herd -b my_branch # Herd a specified branch if it exists, otherwise use default ref
+clowder link -v 0.1 # Set clowder.yaml symlink to a previously saved version
+clowder repo run 'git status' # Run command in .clowder directory
+clowder save 0.1 # Save a version of clowder.yaml with current commit sha's
+clowder start my_feature # Create new branch 'my_feature' for all projects
+clowder stash # Stash changes in all projects
+clowder prune stale_branch # Prune branch 'stale_branch' for all projects
 ```
 
 For more information, see [the docs](http://clowder.readthedocs.io/en/latest/)
 
-# Development
+## Development
 
 See [CONTRIBUTING.md](https://github.com/JrGoodle/clowder/blob/master/CONTRIBUTING.md) for information on setting up your environment for development and contribution guidelines

src/clowder/git/project_repo_impl.py

@@ -13,14 +13,12 @@
 from termcolor import colored
 
 import clowder.util.formatting as fmt
-from clowder.error.clowder_error import ClowderError
 from clowder.error.clowder_git_error import ClowderGitError
 from clowder.git.repo import GitRepo
 from clowder.git.util import (
     existing_git_repository,
     not_detached
 )
-from clowder.util.execute import execute_command
 from clowder.util.file_system import remove_directory
 
 __project_repo_default_ref__ = 'refs/heads/master'
@@ -495,14 +493,16 @@ def _pull(self, remote: str, branch: str) -> None:
         branch_output = fmt.ref_string(branch)
         remote_output = fmt.remote_string(remote)
         self._print(f' - Pull from {remote_output} {branch_output}')
-        command = ['git pull', remote, branch]
-
+        quiet = not self._print_output
         try:
-            execute_command(command, self.repo_path, print_output=self._print_output)
-        except ClowderError:
+            self.repo.git.pull(remote, branch, quiet=quiet)
+        except GitError as err:
             message = colored(' - Failed to pull from ', 'red') + f'{remote_output} {branch_output}'
             self._print(message)
+            self._print(fmt.error(err))
             self._exit(message)
+        except (KeyboardInterrupt, SystemExit):
+            self._exit()
 
     @not_detached
     def _rebase_remote_branch(self, remote: str, branch: str) -> None:
@@ -515,15 +515,16 @@ def _rebase_remote_branch(self, remote: str, branch: str) -> None:
         branch_output = fmt.ref_string(branch)
         remote_output = fmt.remote_string(remote)
         self._print(f' - Rebase onto {remote_output} {branch_output}')
-        command = ['git pull --rebase', remote, branch]
-
+        quiet = not self._print_output
         try:
-            execute_command(command, self.repo_path, print_output=self._print_output)
-        except ClowderError:
+            self.repo.git.pull(remote, branch, rebase=True, quiet=quiet)
+        except GitError as err:
             message = colored(' - Failed to rebase onto ', 'red') + f'{remote_output} {branch_output}'
             self._print(message)
-            self._print(fmt.command_failed_error(command))
+            self._print(fmt.error(err))
             self._exit(message)
+        except (KeyboardInterrupt, SystemExit):
+            self._exit()
 
     def _remote(self, remote: str, remove_dir: bool = False) -> Remote:
         """Get GitPython Remote instance

src/clowder/git/project_repo_recursive.py

@@ -11,9 +11,7 @@
 from termcolor import colored
 
 import clowder.util.formatting as fmt
-from clowder.error.clowder_error import ClowderError
 from clowder.git.project_repo import ProjectRepo
-from clowder.util.execute import execute_command
 
 
 class ProjectRepoRecursive(ProjectRepo):
@@ -123,17 +121,13 @@ def submodule_update_recursive(self, depth: int = 0) -> None:
 
         self._print(' - Recursively update and init submodules')
 
+        error_message = ' - Failed to update submodules'
         if depth == 0:
-            command = ['git', 'submodule', 'update', '--init', '--recursive']
+            self._submodule_command('update', '--init', '--recursive',
+                                    error_msg=error_message)
         else:
-            command = ['git', 'submodule', 'update', '--init', '--recursive', '--depth', depth]
-
-        try:
-            execute_command(command, self.repo_path)
-        except ClowderError:
-            message = colored(' - Failed to update submodules\n', 'red') + fmt.command_failed_error(command)
-            self._print(message)
-            self._exit(message)
+            self._submodule_command('update', '--init', '--recursive', '--depth', depth,
+                                    error_msg=error_message)
 
     def sync(self, fork_remote: str, rebase: bool = False) -> None:
         """Sync fork with upstream remote