Merge branch 'master' into task/updated-documentation

.gitignore

@@ -14,6 +14,7 @@ nosetests.xml
 .coverage
 .tox
 py_ms.egg-info/*
+.eggs/*
 pylintReport.txt
 .scannerwork/
 

.travis.yml

@@ -1,10 +1,10 @@
 language: python
 python:
-    - '3.6'
+  - "3.6"
+  - "3.7"
+  - "3.8"
 install:
-  - pip install pipenv
-  - pip install -U tox coverage==4.0.3 coveralls==1.8.2
-
+  - pip install --upgrade setuptools tox tox-travis coveralls
 script:
   - coverage erase
   - tox

MANIFEST.in

@@ -1,4 +1,8 @@
 include README.md
 include requirements.txt
 include requirements-tests.txt
-recursive-include pyms *
+recursive-include pyms *
+recursive-exclude tests *
+recursive-exclude examples *
+prune tests
+prune examples

Pipfile

@@ -1,36 +1,13 @@
 [[source]]
+name = "pypi"
 url = "https://pypi.org/simple"
 verify_ssl = true
-name = "pypi"
-
-[packages]
-flask = ">=1.1.1"
-python-json-logger = ">=0.1.10"
-pyyaml = ">=5.1.2"
-anyconfig = ">=0.9.8"
-swagger-ui-bundle = ">=0.0.2"
-connexion = {extras = ["swagger-ui"],version = "==2.4.0"}
-jaeger-client = "==4.3.0"
-flask-opentracing = "*"
-opentracing = ">=2.1"
-opentracing-instrumentation = "==3.2.1"
-prometheus_client = ">=0.7.1"
 
 [dev-packages]
-requests-mock = "*"
-coverage = "==4.5.4"
-pytest = "*"
-pytest-cov = "*"
-pylint = "*"
-flake8 = "*"
-tox = "*"
-bandit = "*"
-mkdocs = "*"
-mkdocs-material = "*"
-lightstep = "==4.3.0"
+py-ms = {editable = true,extras = ["tests"],path = "."}
 
-[requires]
-python_version = "3.6"
+[packages]
+py-ms = {editable = true,extras = ["all"],path = "."}
 
-[pipenv]
-allow_prereleases = true
+[requires]
+python_version = "3.7"

README.md

@@ -6,10 +6,24 @@
 [![Requirements Status](https://requires.io/github/python-microservices/pyms/requirements.svg?branch=master)](https://requires.io/github/python-microservices/pyms/requirements/?branch=master)
 [![Total alerts](https://img.shields.io/lgtm/alerts/g/python-microservices/pyms.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/python-microservices/pyms/alerts/)
 [![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/python-microservices/pyms.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/python-microservices/pyms/context:python)
+[![Documentation Status](https://readthedocs.org/projects/py-ms/badge/?version=latest)](https://py-ms.readthedocs.io/en/latest/?badge=latest)
 [![Gitter](https://img.shields.io/gitter/room/DAVFoundation/DAV-Contributors.svg)](https://gitter.im/python-microservices/pyms)
 
 
-PyMS, Python MicroService, is a collections of libraries, best practices and recommended ways to build  microservices with Python.
+PyMS, Python MicroService, is a [Microservice chassis pattern](https://microservices.io/patterns/microservice-chassis.html) 
+like Spring Boot (Java) or Gizmo (Golang). PyMS is a collection of libraries, best practices and recommended ways to build 
+microservices with Python which handles cross-cutting concerns: 
+
+- Externalized configuration
+- Logging
+- Health checks
+- Metrics
+- Distributed tracing
+
+PyMS is powered by [Flask](https://flask.palletsprojects.com/en/1.1.x/), [Connexion](https://github.com/zalando/connexion) 
+and [Opentracing](https://opentracing.io/).
+
+Get started with [Installation](installation.md) and then get an overview with the [Quickstart](quickstart.md).
 
 ## Documentation
 
@@ -25,20 +39,20 @@ nothing to create professional projects. Most articles say:
 - (Sometimes) "Create a swagger specs"
 - "TA-DA! you have a microservice"
 
-But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality?
+But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality? 
 If we have many microservices, what happens with traces?.
 
 There are many problems around Python and microservices and we can`t find anyone to give a solution.
 
-We start creating these projects to try to solve all the problems we have found in our professional lives about
+We start creating these projects to try to solve all the problems we have found in our professional lives about 
 microservices architecture.
 
-Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other felas and friends ;)
+Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other fellas and friends ;) 
 
 ## Installation
 
 ```bash
-pip install py-ms
+pip install py-ms[all]
 ```
 
 # Quickstart
@@ -52,7 +66,7 @@ from flask import jsonify
 
 from pyms.flask.app import Microservice
 
-ms = Microservice(path=__file__) # 1.1
+ms = Microservice() # 1.1
 app = ms.create_app() # 2.1
 
 
@@ -96,6 +110,41 @@ override it.
 
 See [Documentation](https://py-ms.readthedocs.io/en/latest/) to learn more.
 
+## Create a project from scaffold
+
+PyMS has a command line option to create a project template like [Microservices Scaffold](https://github.com/python-microservices/microservices-scaffold).
+This command use [cookiecutter](https://github.com/cookiecutter/cookiecutter) to download and install this [template](https://github.com/python-microservices/microservices-template)
+
+**[Warning]** You must run first `pip install cookiecutter==1.7.0`
+
+```bash
+pyms startproject
+```
+
+this output a lot of options step by step:
+
+```bash
+project_repo_url [https://github.com/python-microservices/microservices-scaffold]: 
+project_name [Python Microservices Boilerplate]: example project
+project_folder [example_project]: 
+project_short_description [Python Boilerplate contains all the boilerplate you need to create a Python package.]: 
+create_model_class [y]: 
+microservice_with_swagger_and_connexion [y]: 
+microservice_with_traces [y]: 
+microservice_with_metrics [y]: 
+application_root [/example_project]: 
+Select open_source_license:
+1 - MIT license
+2 - BSD license
+3 - ISC license
+4 - Apache Software License 2.0
+5 - GNU General Public License v3
+6 - Not open source
+Choose from 1, 2, 3, 4, 5, 6 [1]: 
+```
+
+When you finish to introduce the options, a project will be created in `[project_folder]` folder
+
 ## How To Contrib
 
 We appreciate opening issues and pull requests to make PyMS even more stable & useful! See [This doc](CONTRIBUTING.md)

docs/command_line.md

@@ -0,0 +1,69 @@
+# Commnand line
+
+PyMS has some command to make easy your developments:
+
+```bash
+pyms -h
+```
+Show you a list of options and help instructions to use this command like:
+
+```bash
+usage: main.py [-h] [-v VERBOSE] {encrypt,create-key,startproject} ...
+
+Python Microservices
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -v VERBOSE, --verbose VERBOSE
+                        Verbose
+
+Commands:
+  Available commands
+
+  {encrypt,create-key,startproject}
+    encrypt             Encrypt a string
+    create-key          Generate a Key to encrypt strings in config
+    startproject        Generate a project from https://github.com/python-
+                        microservices/microservices-template
+
+```
+
+## Start a project
+
+Command:
+```bash
+pyms startproject
+```
+
+This command create a project template like [Microservices Scaffold](https://github.com/python-microservices/microservices-scaffold).
+This command use [cookiecutter](https://github.com/cookiecutter/cookiecutter) to download and install this [template](https://github.com/python-microservices/microservices-template)
+
+!!! warning
+    You must run first `pip install cookiecutter==1.7.0`
+
+## Create a key encrypt/decrypt file
+
+Command: 
+```bash
+pyms create-key
+```
+
+Create a key file to encrypt strings in your configuration file. This key is created with [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard).
+You can run the next command in the terminal. See [Encrypt/Decrypt Configuration](encrypt_decryt_configuration.md)
+for more information
+
+## Encrypt a string
+
+Command: 
+```bash
+pyms encrypt [string] 
+```
+
+Encrypt a string to use in your [configfile](configuration.md)
+
+```bash
+pyms encrypt 'mysql+mysqlconnector://important_user:****@localhost/my_schema'
+>>  Encrypted OK: b'gAAAAABeSwBJv43hnGAWZOY50QjBX6uGLxUb3Q6fcUhMxKspIVIco8qwwZvxRg930uRlsd47isroXzkdRRnb4-x2dsQMp0dln8Pm2ySHH7TryLbQYEFbSh8RQK7zor-hX6gB-JY3uQD3IMtiVKx9AF95D6U4ydT-OA=='
+```
+
+See [Encrypt/Decrypt Configuration](encrypt_decryt_configuration.md) for more information

docs/configuration.md

@@ -4,13 +4,14 @@
 
 **CONFIGMAP_FILE**: The path to the configuration file. By default, PyMS search the configuration file in your
 actual folder with the name "config.yml"
-**CONFIGMAP_SERVICE**: the name of the keyword that define the block of key-value of [Flask Configuration Handling](http://flask.pocoo.org/docs/1.0/config/)
-and your own configuration (see the next section to more  info)
+**KEY_FILE**: The path to the key file to decrypt your configuration. By default, PyMS search the configuration file in your
+actual folder with the name "key.key"
 
 ## Create configuration
 Each microservice needs a config file in yaml or json format to work with it. This configuration contains
 the Flask settings of your project and the [Services](services.md). With this way to create configuration files, we 
 solve two problems of the [12 Factor apps](https://12factor.net/):
+
 - Store config out of the code
 - Dev/prod parity: the configuration could be injected and not depends of our code, for example, Kubernetes configmaps
 

docs/encrypt_decryt_configuration.md

@@ -0,0 +1,100 @@
+# Encrypt Configuration
+
+## Configuration
+
+When you work in multiple environments: local, dev, testing, production... you must set critical configuration in your
+variables, like:
+
+config.yml, for local propose:
+```yaml
+pyms:
+  config:
+    DEBUG: true
+    TESTING: true
+    APPLICATION_ROOT : ""
+    SECRET_KEY: "gjr39dkjn344_!67#"
+    SQLALCHEMY_DATABASE_URI: mysql+mysqlconnector://user_of_db:user_of_db@localhost/my_schema
+```
+
+config_pro.yml, for production environment:
+```yaml
+pyms:
+  config:
+    DEBUG: true
+    TESTING: true
+    APPLICATION_ROOT : ""
+    SECRET_KEY: "gjr39dkjn344_!67#"
+    SQLALCHEMY_DATABASE_URI: mysql+mysqlconnector://important_user:****@localhost/my_schema
+```
+
+You can move this file to a [Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/), 
+use [Vault](https://learn.hashicorp.com/vault) or encrypt the configuration with [AWS KMS](https://aws.amazon.com/en/kms/)
+ or [Google KMS](https://cloud.google.com/kms). We strongly recommended this ways to encrypt/decrypt your configuration,
+ but if you want a no vendor locking option or you haven`t the resources to use this methods, we create a way to encrypt
+ and decrypt your variables.
+
+## 1. Generate a key
+PyMS has a command line option to create a key file. This key is created with [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard).
+You can run the next command in the terminal:
+
+```bash
+pyms create-key
+```
+
+Then, type a password and it will create a file called `key.key`. This file contains a unique key. If you loose this file
+and re-run the create command, the key hash will be different and your code encrypted with this key won't be able to be decrypted.
+
+Store the key in a secure site, and NOT COMMIT this key to your repository.
+
+
+## 2. Add your key to your environment
+
+Move, for example, your key to `mv key.key /home/my_user/keys/myproject.key`
+
+then, store this key in a environment variable with:
+
+```bash
+export KEY_FILE=/home/my_user/keys/myproject.key
+```
+
+## 3. Encrypt your information and put in config
+
+Do you remember the example file `config_pro.yml`? Now you can encrypt and decrypt the information, you can run the command
+`pyms encrypt [string]` to generate a crypt string, for example:
+
+```bash
+pyms encrypt 'mysql+mysqlconnector://important_user:****@localhost/my_schema'
+>>  Encrypted OK: b'gAAAAABeSwBJv43hnGAWZOY50QjBX6uGLxUb3Q6fcUhMxKspIVIco8qwwZvxRg930uRlsd47isroXzkdRRnb4-x2dsQMp0dln8Pm2ySHH7TryLbQYEFbSh8RQK7zor-hX6gB-JY3uQD3IMtiVKx9AF95D6U4ydT-OA=='
+```
+
+And put this string in your `config_pro.yml`:
+```yaml
+pyms:
+  config:
+    DEBUG: true
+    TESTING: true
+    APPLICATION_ROOT : ""
+    SECRET_KEY: "gjr39dkjn344_!67#"
+    ENC_SQLALCHEMY_DATABASE_URI: gAAAAABeSwBJv43hnGAWZOY50QjBX6uGLxUb3Q6fcUhMxKspIVIco8qwwZvxRg930uRlsd47isroXzkdRRnb4-x2dsQMp0dln8Pm2ySHH7TryLbQYEFbSh8RQK7zor-hX6gB-JY3uQD3IMtiVKx9AF95D6U4ydT-OA==
+```
+
+Do you see the difference between `ENC_SQLALCHEMY_DATABASE_URI` and `SQLALCHEMY_DATABASE_URI`? In the next step you
+can find the answer
+
+## 4. Decrypt from your config file
+
+Pyms knows if a variable is encrypted if this var start with the prefix `enc_` or `ENC_`. PyMS searchs for your key file
+in the `KEY_FILE` env variable and decrypt this value and store it in the same variable without the `enc_` prefix, 
+por example, 
+
+```yaml
+ENC_SQLALCHEMY_DATABASE_URI: gAAAAABeSwBJv43hnGAWZOY50QjBX6uGLxUb3Q6fcUhMxKspIVIco8qwwZvxRg930uRlsd47isroXzkdRRnb4-x2dsQMp0dln8Pm2ySHH7TryLbQYEFbSh8RQK7zor-hX6gB-JY3uQD3IMtiVKx9AF95D6U4ydT-OA==
+```
+
+Will be stored as 
+
+```bash
+SQLALCHEMY_DATABASE_URI: mysql+mysqlconnector://user_of_db:user_of_db@localhost/my_schema
+```
+
+And you can access to this var with `current_app.config["SQLALCHEMY_DATABASE_URI"]`

docs/examples.md

@@ -1,7 +1,7 @@
 # Examples
 
 ```bash
-pip install py-ms
+pip install py-ms[all]
 ```
 
 config.yml:

docs/index.md

@@ -41,8 +41,10 @@ Nowadays, is not perfect and we have a looong roadmap, but we hope this library
 * [Quickstart](quickstart.md)
 * [Configuration](configuration.md)
 * [Services](services.md)
-* [PyMS structure](structure.md)
 * [Microservice class](ms_class.md)
-* [Examples](examples.md)
 * [Routing](routing.md)
+* [Encrypt/Decrypt Configuration](encrypt_decryt_configuration.md)
+* [Command line](command_line.md)
+* [Examples](examples.md)
+* [PyMS structure](structure.md)
 * [Structure of a microservice project](structure_project.md)