Artman Refactor

.coveragerc

@@ -1,12 +1,13 @@
 [run]
 branch = True
-source = pipeline/
+source = artman/
 
 [report]
+# fail_under = 75
 show_missing = True
 exclude_lines =
     # Re-enable the standard pragma
     pragma: no-cover
-    
+
     # Ignore debug-only repr
     def __repr__

.gitignore

@@ -46,6 +46,7 @@ pip-delete-this-directory.txt
 # Unit test / coverage reports
 htmlcov/
 .tox/
+.nox/
 .coverage
 .coverage.*
 .cache

.travis.yml

@@ -4,18 +4,18 @@ sudo: false
 matrix:
   include:
     - python: 2.7
-      env: TOXENV=py27
+      env: PY_VERSION='2.7'
     - python: 3.4
-      env: TOXENV=py34
+      env: PY_VERSION='3.4'
     - python: 3.5
-      env: TOXENV=py35
-    - python: 3.5
-      env: TOXENV=lint
+      env: PY_VERSION='3.5'
+    - python: 3.6
+      env: PY_VERSION='3.6'
 cache:
   directories:
     - ${HOME}/.cache
 install:
-  - pip install --upgrade tox
-script: tox
+  - pip install --upgrade nox-automation codecov
+script: nox -e "unit_tests(python_version='$PY_VERSION')"
 after_success:
   - codecov

CONTRIBUTING.rst

@@ -34,12 +34,14 @@ requests. Fork the repository, and make your changes in the forked repository.
 Include tests
 -------------
 
-Be sure to add relevant tests and run then them using :code:`tox` before making the pull request.
+Be sure to add relevant tests and run then them using :code:`nox` befor
+ making the pull request.
+
 Docs will be updated automatically when we merge to `master`, but
-you should also build the docs yourself via :code:`tox -e docs`, making sure that the docs build OK
-and that they are readable.
+you should also build the docs yourself via :code:`nox -e docs`, making sure
+that the docs build OK and that they are readable.
 
-.. _`tox`: https://tox.readthedocs.org/en/latest/
+.. _`nox`: https://nox.readthedocs.io/en/latest/
 
 Make the pull request
 ---------------------
@@ -56,7 +58,8 @@ Using a Development Checkout
 You’ll have to create a development environment to hack on
 `artman`_, using a Git checkout:
 
--   While logged into your GitHub account, navigate to the `artman repo`_ on GitHub.
+-   While logged into your GitHub account, navigate to the
+    `artman repo`_ on GitHub.
 -   Fork and clone the `artman` repository to your GitHub account
     by clicking the "Fork" button.
 -   Clone your fork of `artman` from your GitHub account to your
@@ -81,21 +84,19 @@ You’ll have to create a development environment to hack on
 Now your local repo is set up such that you will push changes to your
 GitHub repo, from which you can submit a pull request.
 
--   Create use tox to create development virtualenv in which `artman`_ is installed:
-
   .. code:: bash
 
-    sudo pip install tox
-    cd ~/hack-on-artman
-    tox -e devenv
+    sudo pip install virtualenv virtualenvwrapper
+    mkvirtualenv --python=`which python3` artman
+    pip install -e .
 
--   This is creates a tox virtualenv named `development` that has artman installed.
+-   This is creates a virtualenv named `artman` that has artman installed.
     Activate it to use artman locally, e.g, from the python prompt.
 
   .. code:: bash
 
     cd ~/hack-on-artman
-    . ./tox/develop/bin/activate
+    workon artman
 
 .. _`artman`: https://github.com/googleapis/artman
 .. _`artman repo`: https://github.com/googleapis/artman
@@ -105,19 +106,19 @@ Running Tests
 -------------
 
 -   To run the full set of `artman` tests on all platforms, install
-    `tox`_ into a system Python.  The :code:`tox` console script will be
+    `nox`_ into a system Python.  The :code:`nox` console script will be
     installed into the scripts location for that Python.  While in the
-    `artman` checkout root directory (it contains :code:`tox.ini`),
-    invoke the `tox` console script.  This will read the :code:`tox.ini` file and
-    execute the tests on multiple Python versions and platforms; while it runs,
-    it creates a virtualenv for each version/platform combination.  For
+    `artman` checkout root directory (it contains :code:`nox.py`),
+    invoke the `nox` console script.  This will read the :code:`nox.py` file
+    and execute the tests on multiple Python versions and platforms; while
+    it runs, it creates a virtualenv for each version/platform combination. For
     example:
 
   .. code:: bash
 
-      sudo pip install tox
+      sudo pip install nox-automation
       cd ~/hack-on-artman
-      tox
+      nox
 
 Contributor License Agreements
 ------------------------------

README.rst

@@ -19,34 +19,52 @@ from the protobuf source IDL and additional configuration in YAML files.
 Installation
 ------------
 
-At the moment, this library is under development, so please see
-`using a development checkout`_ for installation instructions
+You can now install artman directly from pip:
 
-.. _`using a development checkout`: https://github.com/googleapis/artman/blob/master/CONTRIBUTING.rst#using-a-development-checkout
+.. code::
+
+    pip install googleapis-artman
+
+You may need root privileges if you are not installing inside of a virtualenv.
+This will make the ``artman`` command available on your system.
 
 
 Usage
 -----
 
-Currently, this tool can only be run in the development environment of the
-development team, as some dependencies are yet to be published. See USAGE_ for
-details.
+Before you can use artman, you will need a configuration file. You can run
+``artman init`` to create a simple configuration file.
+
+For building a GAPIC (the most common task), the usage looks like:
+
+.. code::
+
+    artman build --api pubsub --language python
+
+This assumes that you have checkouts of both `googleapis`_ and `toolkit`_
+on your system (and that toolkit is able to run; e.g. you need Java).
+
+.. _`googleapis`: https://github.com/googleapis/googleapis
+.. _`toolkit`: https://github.com/googleapis/toolkit
 
-.. _USAGE: https://github.com/googleapis/artman/blob/master/USAGE.rst
+Artman also takes a ``--publish`` argument to decide where to stage the
+code. Using ``--publish github`` will create a pull request on GitHub
+automatically.
 
 
 Python Versions
 ---------------
 
-artman is currently tested with Python 2.7.
+artman is currently tested with Python 2.7 and Python 3.4.
 
 
 Contributing
 ------------
 
 Contributions to this library are always welcome and highly encouraged.
 
-See the `CONTRIBUTING`_ documentation for more information on how to get started.
+See the `CONTRIBUTING`_ documentation for more information on how to get
+started.
 
 .. _`CONTRIBUTING`: https://github.com/googleapis/artman/blob/master/CONTRIBUTING.rst
 
@@ -66,7 +84,8 @@ stable.
 Details
 -------
 
-For detailed documentation of the modules in artman, please watch `DOCUMENTATION`_.
+For detailed documentation of the modules in artman, please watch
+`DOCUMENTATION`_.
 
 .. _`DOCUMENTATION`: https://googleapis-artman.readthedocs.org/
 

pipeline/utils/__init__.py → artman/cli/__init__.py

@@ -1,13 +1,21 @@
-# Copyright 2016 Google Inc. All Rights Reserved.
+# Copyright 2017 Google
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-# http://www.apache.org/licenses/LICENSE-2.0
+#    http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
+from __future__ import absolute_import, unicode_literals
+
+from artman.cli import configure
+from artman.cli import main
+from artman.cli import support
+
+__all__ = ('configure', 'main', 'support')

artman/cli/configure.py

@@ -0,0 +1,144 @@
+# Copyright 2017 Google
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from copy import copy
+import getpass
+import importlib
+import io
+import logging
+import os
+
+import six
+
+from ruamel import yaml
+
+from artman.utils.logger import logger
+from artman.utils.logger import setup_logging
+
+__all__ = ('configure',)
+
+
+def configure(log_level=logging.INFO):
+    """Allow the user to write a new configuration file.
+
+    Returns:
+        int: An exit status.
+    """
+    user_config = {}
+
+    # Walk the user through basic configuration.
+    setup_logging(log_level)
+    logger.info('Welcome to artman. We will get you configured.')
+    logger.info('When this is done, config will be stored in ~/.artman.yaml.')
+    logger.info('')
+
+    # Go through each step.
+    # These are split out to make testing them easier.
+    user_config['local_paths'] = _configure_local_paths(
+        user_config.get('local_paths', {}),
+    )
+    user_config['publish'] = _configure_publish()
+    if user_config['publish'] == 'github':
+        user_config['github'] = _configure_github(
+            user_config.get('github', {}),
+        )
+
+    # Write the final configuration.
+    config_yaml = yaml.dump(user_config,
+        block_seq_indent=2,
+        default_flow_style=False,
+        indent=2,
+    )
+    with io.open(os.path.expanduser('~/.artman.yaml'), 'w+') as file_:
+        file_.write('---\n')
+        file_.write(config_yaml)
+    logger.success('Configuration written successfully to ~/.artman.yaml.')
+
+
+def _configure_local_paths(local_paths):
+    """Return a copy of user_config with local_paths set.
+
+    Args:
+        local_paths (dict): The starting local_paths portion ofuser config.
+
+    Returns:
+        dict: The new local_paths dictionary.
+    """
+    answer = copy(local_paths)
+
+    # Ask the user for a repository root.
+    while not answer.get('reporoot'):
+        logger.info('First, we need to know where you store most code on your '
+                    'local machine.')
+        logger.info('Other paths will derive from this, but most are '
+                    'individually configurable.')
+        logger.info('Note: Use of ~ is fine here.')
+        answer['reporoot'] = six.moves.input('Local code path: ')
+        answer['reporoot'] = answer['reporoot'].rstrip('/').strip()
+
+    # Set up dependent directories.
+    reporoot = answer['reporoot']
+    for dep in ('api-client-staging', 'googleapis', 'toolkit'):
+        location = six.moves.input(
+            'Path for {0} (default: {1}/{0}): '.format(dep, reporoot)
+        ).rstrip('/').strip()
+        if location:
+            answer[dep.replace('-', '_')] = location
+
+    # Done; return the answer.
+    return answer
+
+
+def _configure_publish(publish=None):
+    """Determine and return the default publisher.
+
+    Args:
+        publish (str): The current default publisher (may be None).
+
+    Returns:
+        str: The new default publisher.
+    """
+    # Set up publishing defaults.
+    logger.info('Where do you want to publish code by default?')
+    logger.info('The common valid options are "github" and "local".')
+    publish = six.moves.input('Default publisher: ').lower()
+    try:
+        importlib.import_module('artman.tasks.publish.%s' % publish)
+        return publish
+    except ImportError:
+        logger.error('Invalid publisher.')
+        return _configure_publish()
+
+
+def _configure_github(github):
+    """Determine and return the GitHub configuration.
+
+    Args:
+        github (dict): The current GitHub configuration.
+
+    Returns:
+        dict: The new GitHub configuration.
+    """
+    answer = copy(github)
+    logger.info('Since you intend to publish to GitHub, you need to '
+                'supply credentials.')
+    logger.info('Create an access token at: '
+                'https://github.com/settings/tokens')
+    logger.info('It needs the "repo" scope and nothing else.')
+    while not answer.get('username'):
+        answer['username'] = six.moves.input('GitHub username: ')
+    while not answer.get('token'):
+        answer['token'] = getpass.getpass('GitHub token (input is hidden): ')
+    return answer