Skip to content

Commit

Permalink
Added more params
Browse files Browse the repository at this point in the history
* Added pid and env-file
* Added docs for creating new transformers and parameters
  • Loading branch information
micahhausler committed May 30, 2016
1 parent 05e4d36 commit 62ac663
Show file tree
Hide file tree
Showing 11 changed files with 114 additions and 10 deletions.
6 changes: 6 additions & 0 deletions container_transform/marathon.py
Original file line number Diff line number Diff line change
Expand Up @@ -373,3 +373,9 @@ def emit_net_mode(self, net_mode):

def emit_user(self, user):
return [{'key': 'user', 'value': user}]

def emit_pid(self, pid):
return [{'key': 'pid', 'value': pid}]

def emit_env_file(self, env_file):
return [{'key': 'env-file', 'value': ef} for ef in env_file]
42 changes: 39 additions & 3 deletions container_transform/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ class OutputTransformationTypes(Enum):
'required': False,
},
TransformationTypes.SYSTEMD.value: {
'name': 'mem_limit',
'name': 'memory',
'required': False,
},
TransformationTypes.MARATHON.value: {
Expand Down Expand Up @@ -446,6 +446,42 @@ class OutputTransformationTypes(Enum):
'name': 'container.docker.parameters.user',
'required': False
},
}
# TODO: 'containername', 'pid'
},
'env_file': {
TransformationTypes.ECS.value: {
'name': None,
'required': False
},
TransformationTypes.COMPOSE.value: {
'name': 'env_file',
'required': False,
},
TransformationTypes.SYSTEMD.value: {
'name': 'env-file',
'required': False,
},
TransformationTypes.MARATHON.value: {
'name': 'container.docker.parameters.env-file',
'required': False,
'type': list,
},
},
'pid': {
TransformationTypes.ECS.value: {
'name': None,
'required': False
},
TransformationTypes.COMPOSE.value: {
'name': 'pid',
'required': False,
},
TransformationTypes.SYSTEMD.value: {
'name': 'pid',
'required': False,
},
TransformationTypes.MARATHON.value: {
'name': 'container.docker.parameters.pid',
'required': False,
},
},
})
8 changes: 6 additions & 2 deletions container_transform/systemd.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@
--name {{ name }} \\
{%- if cpu_shares %}
--cpu {{ cpu_shares }} \\{% endif -%}
{% if mem_limit %}
--memory {{ mem_limit }} \\{% endif -%}
{% if memory %}
--memory {{ memory }} \\{% endif -%}
{% if hostname %}
--hostname {{ hostname }} \\{% endif -%}
{% if pid %}
--pid {{ pid }} \\{% endif -%}
{% if entrypoint %}
--entrypoint {{ entrypoint }} \\{% endif -%}
{% for port in ports %}
Expand All @@ -45,6 +47,8 @@
--label {{ label[0] }}="{{ label[1] }}" \\{% endfor -%}{% endif -%}
{% for link in links %}
--link {{ link }} \\{% endfor -%}
{% for ef in env_file %}
--env-file {{ ef }} \\{% endfor -%}
{% for vf in volumes_from %}
--volumes-from {{ vf }} \\{% endfor -%}
{% for ns in dns %}
Expand Down
2 changes: 2 additions & 0 deletions container_transform/tests/composev2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ services:
ports:
- "5000:5000"
- "53:53/udp"
pid: "host"
env_file: "./web.env"
volumes:
- .:/code
labels:
Expand Down
1 change: 1 addition & 0 deletions container_transform/tests/composev2_output.service
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ ExecStartPre=-/usr/bin/docker rm web
ExecStartPre=/usr/bin/docker pull <image>
ExecStart=/usr/bin/docker run \
--name web \
--pid host \
-p 5000:5000 \
-p 53:53/udp \
-v .:/code \
Expand Down
4 changes: 1 addition & 3 deletions container_transform/tests/systemd_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ def setUp(self):
self.transformer = SystemdTransformer()

def test_emit_containers(self):
self.maxDiff = None
containers = [{
'command': 'celery worker',
'environment': {
Expand All @@ -27,7 +26,7 @@ def test_emit_containers(self):
},
'image': 'me/myapp',
'links': ['db', 'redis', 'web'],
'mem_limit': '67108864b',
'memory': '67108864b',
'name': 'worker',
'ports': [
'8000',
Expand All @@ -48,7 +47,6 @@ def test_emit_containers(self):
}
}
}]

service_file = '{}/worker.service'.format(os.path.dirname(__file__))
service_contents = open(service_file).read()
self.assertEqual(
Expand Down
16 changes: 16 additions & 0 deletions container_transform/transformer.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
'domain': list,
'labels': dict,
'network': list,
'env-file': list,
'pid': str,
}


Expand Down Expand Up @@ -160,6 +162,20 @@ def ingest_labels(self, labels):
def emit_labels(self, labels):
return labels

def ingest_pid(self, pid):
return pid

def emit_pid(self, pid):
return pid

def ingest_env_file(self, env_file):
if not isinstance(env_file, list) and env_file is not None:
env_file = [env_file]
return env_file

def emit_env_file(self, env_file):
return env_file

@abstractmethod
def ingest_port_mappings(self, port_mappings):
raise NotImplementedError
Expand Down
2 changes: 1 addition & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ clean:
rm -rf $(BUILDDIR)/*

html:
$(SPHINXBUILD) -W -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
$(SPHINXBUILD) -n -W -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

Expand Down
39 changes: 38 additions & 1 deletion docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ To get the source source code and run the unit tests, run::
virtualenv env
. env/bin/activate
pip install -e .[all]
python setup.py install
python setup.py nosetests

While 100% code coverage does not make a library bug-free, it significantly
Expand Down Expand Up @@ -63,6 +62,44 @@ When in the project directory::
python setup.py build_sphinx
open docs/_build/html/index.html

Adding a new parameter
----------------------

If there is a docker parameter that you'd like to add support for, here's a
quick overview of what is involved:

1. Make a new parameter in the ``ARG_MAP`` in the file ``container-transform/schema.py``
2. Check what the parameter name is for each supported transformation type.
There are links to the documentation for each type on the :doc:`usage` page
3. Create an ``ingest_<param>`` and ``emit_<param>`` method on the
:class:`BaseTransformer<container_transform.transformer.BaseTransformer>` class
4. Add any data transformations by overriding the base methods that each format
requires.
5. Add tests to cover any new logic. Don't just use a client test to make
coverage 100%

Adding a new Transformer
------------------------

If you'd like to add a new format, please create an issue before making a pull
request in order to discuss any major design decisions before putting in
valuable time writing the actual code.

Below is a rough checklist of creating a new transformer type:

* Create a file and class in the base :py:mod:`container_transform` module
* Implement all abstract methods on the :class:`BaseTransformer<container_transform.transformer.BaseTransformer>`
class
* Add the class to the ``TRANSFORMER_CLASSES`` in the ``converter.py`` file.
* Add the type to the enums at the top of teh ``schema.py`` file.
* Add a key to each of the dictionaries in the ``ARG_MAP`` parameters
* If a docker parameter is not supported in your transformer, still create
a dictionary for it, but set the name to ``None``
* Create a test file in the tests module for your transformer. Try to get at
least 90% coverage of your transformer before adding any tests to the
:py:mod:`client_tests.py<container_transform.tests.client_test>` module.
* Add client tests just to make sure the command doesn't blow up

Release Checklist
-----------------

Expand Down
2 changes: 2 additions & 0 deletions docs/release_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ v1.1.2
------

* Fixed udp port handling
* Added support for ``pid`` and ``env-file`` parameters
* Added docs for adding parameters and creating new transformers

v1.1.1
------
Expand Down
2 changes: 2 additions & 0 deletions docs/usage.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _usage:

Usage
=====

Expand Down

0 comments on commit 62ac663

Please sign in to comment.