Reviewed README.md.

decryptus · Aug 27, 2019 · bd98dc9 · bd98dc9
1 parent 8d0496a
commit bd98dc9
Show file tree

Hide file tree

Showing 9 changed files with 327 additions and 30 deletions.
diff --git a/README.md b/README.md
@@ -1,11 +1,295 @@
-## Installation
+## auton project
 
-pip install auton
+auton is a free and open-source, we develop it to run programs and command-lines on remote servers through HTTP protocol.
+There are two programs, auton for client side and autond for server side.
+auton is just a helper to transform command-lines into HTTP protocol, it is able to transform basic arguments, file arguments and environment variables.
+For example, you can use auton from CI/CD to run on remote servers, you just need to configure your endpoints:
+  - [ansible](https://github.com/ansible/ansible)
+  - [curl](https://github.com/curl/curl)
+  - [terraform](https://github.com/hashicorp/terraform)
 
-### Running daemon
+## Quickstart
 
-`autond -c <conffile> -p <pidfile> --logfile <logfile>`
+Using autond in Docker
 
-### Running foreground
+`docker-compose up -d`
 
-`autond -f -c <conffile> -p <pidfile> --logfile <logfile>`
+See [docker-compose.yml](docker-compose.yml)
+
+## Environment variables
+
+### autond
+
+| Variable         | Description                 | Default |
+|:-----------------|:----------------------------|:--------|
+| `AUTOND_CONFIG`  | Configuration file contents<br />(e.g. `export AUTOND_CONFIG="$(cat auton.yml)"`) |  |
+| `AUTOND_LOGFILE` | Log file path               | /var/log/autond/daemon.log |
+| `AUTOND_PIDFILE` | autond pid file path        | /run/auton/autond.pid |
+| `AUTON_GROUP`    | auton group                 | auton or root |
+| `AUTON_USER`     | auton user                  | auton or root |
+
+### auton
+
+| Variable               | Description                 | Default |
+|:-----------------------|:----------------------------|:--------|
+| `AUTON_AUTH_USER`      | user for authentication     |  |
+| `AUTON_AUTH_PASSWD`    | password for authentication |  |
+| `AUTON_ENDPOINT`       | name of endpoint            |  |
+| `AUTON_LOGFILE`        | Log file path               | /var/log/auton/auton.log |
+| `AUTON_NO_RETURN_CODE` | Do not exit with return code if present | False |
+| `AUTON_UID`            | auton job uid               | random uuid |
+| `AUTON_URI`            | autond uri(s)<br />(e.g. http://auton-01.example.org:8666,http://auton-02.example.org) |  |
+
+## Autond configuration
+
+See configuration example [etc/auton/auton.yml.example](etc/auton/auton.yml.example)
+
+### Endpoints
+
+In this example, we declared three endpoints: ansible-playbook-ssh, ansible-playbook-http, curl.
+They used subproc plugin.
+
+```yaml
+endpoints:
+  ansible-playbook-ssh:
+    plugin: subproc
+    config:
+      prog: ansible-playbook
+      timeout: 3600
+      args:
+        - '/etc/ansible/playbooks/ssh-install.yml'
+        - '--tags'
+        - 'sshd'
+      become:
+        enabled: true
+      env:
+        DISPLAY_SKIPPED_HOSTS: 'false'
+  ansible-playbook-http:
+    plugin: subproc
+    config:
+      prog: ansible-playbook
+      timeout: 3600
+      args:
+        - '/etc/ansible/playbooks/http-install.yml'
+        - '--tags'
+        - 'httpd'
+      become:
+        enabled: true
+      env:
+        DISPLAY_SKIPPED_HOSTS: 'false'
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      timeout: 3600
+```
+
+### Authentication
+
+To enable authentication, you must add `auth_basic` and `auth_basic_file` lines in section general:
+
+```yaml
+general:
+  auth_basic:      'Restricted'
+  auth_basic_file: '/etc/auton/auton.passwd'
+```
+
+To generate `auth_basic_file` use `htpasswd`:
+`htpasswd -c -s /etc/auton/auton.passwd foo`
+
+And you have to add for each modules route `auth: true`:
+
+```yaml
+modules:
+  job:
+    routes:
+      run:
+        handler:   'job_run'
+        regexp:    '^run/(?P<endpoint>[^\/]+)/(?P<id>[a-z0-9][a-z0-9\-]{7,63})$'
+        safe_init: true
+        auth:      true
+        op:        'POST'
+      status:
+        handler:   'job_status'
+        regexp:    '^status/(?P<endpoint>[^\/]+)/(?P<id>[a-z0-9][a-z0-9\-]{7,63})$'
+        auth:      true
+        op:        'GET'
+```
+
+You can use section `users` to specify users allowed by endpoint:
+```yaml
+  ansible-playbook-ssh:
+    plugin: subproc
+    users:
+      maintainer: true
+      bob: true
+    config:
+      prog: ansible-playbook
+      timeout: 3600
+      args:
+        - '/etc/ansible/playbooks/ssh-install.yml'
+        - '--tags'
+        - 'sshd'
+      become:
+        enabled: true
+      env:
+        DISPLAY_SKIPPED_HOSTS: 'false'
+```
+
+#### Plugin subproc
+
+subproc plugin execute subprocess `proc`:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+```
+
+You can also use section `become` to execute with an other user:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      become:
+        enabled: true
+        user: foo
+```
+
+You can also use keyword `timeout` to raise an exception after n seconds (default: 60 seconds):
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      timeout: 3600
+```
+
+You can also use section `args` to define arguments always present:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      args:
+        - '-s'
+        - '-4'
+```
+
+You can also use keyword `disallow-args` to disable args from client:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      args:
+        - '-vvv'
+        - 'https://example.com'
+      disallow-args: true
+```
+
+You can also use section `argfiles` to define arguments files always present:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      argfiles:
+        - arg: '--key'
+          filepath: /tmp/private_key
+        - arg: '-d@'
+          filepath: /tmp/data
+```
+
+You can also use keyword `disallow-argfiles` to disable arguments files from client:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      argfiles:
+        - arg: '--key'
+          filepath: /tmp/private_key
+        - arg: '-d@'
+          filepath: /tmp/data
+      disallow-argfiles: true
+```
+
+You can also use section `env` to define environment variables always present:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      env:
+        HTTP_PROXY: http://proxy.example.com:3128/
+        HTTPS_PROXY: http://proxy.example.com:3128/
+```
+
+You can also use keyword `disallow-env` to disable environment variables from client:
+```yaml
+endpoints:
+  curl:
+    plugin: subproc
+    config:
+      prog: curl
+      env:
+        HTTP_PROXY: http://proxy.example.com:3128/
+        HTTPS_PROXY: http://proxy.example.com:3128/
+      disallow-env: true
+```
+
+Predefined AUTON environment variables during execution:
+
+| Variable           | Description                                   |
+|:-------------------|:----------------------------------------------|
+| `AUTON`            | Mark the job is executed in AUTON environment |
+| `AUTON_JOB_TIME`   | Current time in local time zone               |
+| `AUTON_JOB_GMTIME` | Current time in GMT                           |
+| `AUTON_JOB_UID`    | Current job uid passed from client            |
+| `AUTON_JOB_UUID`   | Unique ID of the current job                  |
+
+## Auton command-lines
+
+#### endpoint curl examples:
+
+Simple call url https://example.com:
+
+`auton --endpoint curl --uri http://localhost:8666 -a 'https://example.com'`
+
+You can also add environment variable HTTP\_PROXY:
+
+`auton --endpoint curl --uri http://localhost:8666 -a 'https://example.com' -e 'HTTP_PROXY=http://proxy.example.com:3128/'`
+
+You can also load environment variables from local files:
+
+`auton --endpoint curl --uri http://localhost:8666 -a 'https://example.com' --load-envfile foo.env`
+
+You can also tell to autond to load environment variables files from its fs:
+
+`auton --endpoint curl --uri http://localhost:8666 -a 'https://example.com' --envfile /etc/auton/auton.env`
+
+You can also add multiple autond uris for high availability:
+
+`auton --endpoint curl --uri http://localhost:8666 --uri http://localhost:8667 -a 'https://example.com'`
+
+You can also add arguments files to send local files:
+
+`auton --endpoint curl --uri http://localhost:8666 -A '--cacert=cacert.pem' -a 'https://example.com'`
+
+You can also add multiple arguments:
+
+`auton --endpoint curl --uri http://localhost:8666 --multi-args '-vvv -u foo:bar https://example.com' --multi-argsfiles '-d@=foo -d@=bar --cacert=cacert.pem'`
+
+You can also get file content from stdin with `-`:
+
+`cat foo | auton --endpoint curl --uri http://localhost:8666 --multi-argsfiles '--key=private_key --pubkey=public_key -T=-' --multi-args '-vvv -u foo:bar sftp://example.com'`
diff --git a/auton/plugins/subproc.py b/auton/plugins/subproc.py
@@ -136,13 +136,13 @@ def _mk_argfiles(self, args, cargfiles, pargfiles):
                     LOG.error("invalid filepath in configuration argfiles for target: %r", self.target.name)
                     return None
 
-                if cargfile['arg'].startswith('@'):
+                if cargfile['arg'].endswith('@'):
                     if len(cargfile['arg']) == 1:
                         LOG.error("invalid arg %r in configuration argfiles for target: %r",
                                   cargfile['arg'],
                                   self.target.name)
                         return None
-                    r.extend([cargfile['arg'][1:], "@%s" % cargfile['filepath']])
+                    r.extend([cargfile['arg'][:-1], "@%s" % cargfile['filepath']])
                 else:
                     r.extend([cargfile['arg'], cargfile['filepath']])
 
@@ -163,13 +163,13 @@ def _mk_argfiles(self, args, cargfiles, pargfiles):
                     filepath = os.path.join(tmpdir, pargfile['filename'])
                 helpers.base64_decode_file(StringIO(pargfile['content']),
                                            filepath)
-                if pargfile['arg'].startswith('@'):
+                if pargfile['arg'].endswith('@'):
                     if len(pargfile['arg']) == 1:
                         LOG.error("invalid arg %r in payload argfiles for target: %r",
                                   pargfile['arg'],
                                   self.target.name)
                         return None
-                    r.extend([pargfile['arg'][1:], "@%s" % filepath])
+                    r.extend([pargfile['arg'][:-1], "@%s" % filepath])
                 else:
                     r.extend([pargfile['arg'], filepath])
 
@@ -196,7 +196,7 @@ def _mk_env(self, cenv, fenv, penv, ovars):
 
         if fenv:
             if not isinstance(fenv, list):
-                LOG.warning("invalid configuration envfiles for target: %r", self.target.name)
+                LOG.warning("invalid payload envfiles for target: %r", self.target.name)
                 return r
 
             for key, val in self._load_envfile(fenv).iteritems():
@@ -281,6 +281,8 @@ def do_run(self, obj):
         texit   = threading.Event()
         proc    = None
 
+        LOG.debug("cmd line: %r", bargs + args)
+
         try:
             proc  = subprocess.Popen(bargs + args,
                                      stdout = subprocess.PIPE,