Add environment variables in molecule.yml #707
Comments
retr0h
added a commit
to retr0h/molecule
that referenced
this issue
Jan 12, 2017
Allows the embedding of environment variables inside molecule.yml. Originally requested in ansible#643. Fixes: ansible#707
retr0h
added a commit
that referenced
this issue
Jan 12, 2017
|
Implemented by #709. |
retr0h
added a commit
that referenced
this issue
Jun 12, 2017
* Init add of molecule v2 work Initial work of a v2 based Molecule. This version of Molecule has a slightly different config, uses Ansible to boostrap the container vs relying on Molecule glue drivers, and only supports Docker at this time. Implemented the following subcommands. 1. Create 2. Converge 3. Destroy 4. Verify 5. Lint Will no longer squash into this commit, and will build ontop. * Various fixups on way to MVP * Reworked provider class to handle converge. * Hid the use of AnsiblePlaybook inside the provisioner. * Passing of os.environ to sh bake calls. * Continue to cleanup tests. * Improved coverage * Added unit tests for docker driver. * 100% ansible.py provisioner. * 100% testinfra.py verifier. * Added multi-node scenario The platform name is now appended with the scenario name. * Added basic functional testing framework Added very basic functional testing from v1 molecule. * Added init and enabled lint subcommand Added a basic init subcommand and enabled the existing lint command. * Moved config properties into respective base Config didn't need properties for everything, since base has properties for each top level class. Moved the properties into their respective class. * Added test and converge sequences * Only test with ansible 2.2 * Many cleanups * Cleaned up tox.ini. * Added a dependency sub command. * Added gilt dependency manager. * Removed crazy parametrized test config handling. * Init cleanups * Removed duplicate _process_templates calls. * Updated docstring to match rest of project. * Added doc strings where necessary Did not document methods which are obvious. * Added usage/configuration through doc strings As part of documenting ansible-lint, implemented proper exclude options handling. * syntax and test tweaks (#675) * [v2] move cookiecutter back to molecule directory (#678) - also skip guilt tests verifying bake params, since the param ordering is not deterministic * Implemented scenario_name filtering sub commands (#681) Sub commands will execute all scenarios, unless the `--scenario-name` flag is passed. Fixes: #680 * No merging of local config onto scenario config (#685) It no longer makes sense to merge the local config onto the scenario config since a scenario can be anything. In an attempt to keep things simple, removed this functionality. Partial-Fix: #684 * Implemented duplicate scenario name validation (#686) Since we can target a scenario based on name, we should not allow duplicate scenarios to be created. Fixes: #682 * Corrected a flake8 error * Implemented yaml inventory file (#690) Moved the generated host inventory from the proprietary "ini" format to Ansible's new yaml format. Fixes: #683 * Moved vivify under provisioner class Until we need it elsewhere lets keep it private and inside the class. * Add idempotence sub command (#691) Fixes: #677 * Bumped license year (#692) * Corrected a few rogue double quotes * Added state module from v1 Ported over v1's state module. * Bind a state instance onto each config instance * Add state management to destroy, create, converge The destroy, create, and converge sub commands are logging their state to the state file. We will then make decisions off of that data later. * Idempotence errors if not already converged (#698) Idempotence should only be run when instances have already been converged. Fixes: #694 * Added idempotence to test sequence * Run dependency managers only when config present (#699) Molecule should automatically install dependencies, when the requirements file is present. Fixes: #695 * Added dependency to test sequence * Added syntax sub command (#700) Fixes: #697 * Implemented a check command (#701) Fixes: #696 * Documented check and converge subcommands deps * Added full quickstart output * Corrected skipping testinfra tests Tests were skipped due to bad inventory path. Also, updated cookiecutter `molecule init` template. * Implemented status subcommand (#706) * Better capsys variable naming * Implemented status subcommand Fixes: #703 * Implemented Molecule ansible plugin paths (#708) Implemented a directory structure to hold ansible plugins for use with ansible-playbook. In this commit, added a filter plugin to construct $instance_name-$scenario_name in playbooks. * Use safe_dump and reuse existing helper methods * Make ansible.cfg settings configurable Ansible.cfg is written to disk from the provisioners config options. File is entirely customizable. * Added string interpolation to molecule.yml (#709) Allows the embedding of environment variables inside molecule.yml. Originally requested in #643. Fixes: #707 * Corrected minor flake8 issue * Corrected molecule error in dev env (#710) When developing molecule, any molecule command will fail when executing from the project root, since os_walk detects the cookiecutter templates as a scenario. Updated with an intelligent glob to find scenarios. Fixes: #705 * Point to v2 docs on the v2 branch * Added warning about pip install on v2 * Corrected flake8 errors * Implemented a login subcommand (#711) * Cleaned up interpolation coverage When moving code over from Docker, there was uncovered code. Deleted the code, since I don't see a valid reason we would encounter the raise. * Implemented a login subcommand Ported v1 login to v2. Fixes: #704 * Port molecule v1 host/group vars handling (#712) Fixes: #693 * Updated provisioner's docs Added a couple features which needed documented. Bad me. * Added basic Config module documentation * Added sysexit_with_message Added a method for exit and error message handling. * Throw an error if config contains wrong names Ensure we can only configure molecule with supported configurations. * Added Children inventory creation In implementing this feature broke out platforms into own class. This keeps all top level config options as classes. * Brought in State autodocs * doco * Corrected doc line length * Status reports kitchen like output * Renamed Status to List * Added Ansible 2.2.1 as supported * Corrected coverage * Use state file to track status * Slightly improved output * Initial py36 support * Officially support Ansible 2.2.1 With CVE-2016-9587[1] decided to only support versions of Ansible which were fixed. Not supporting 2.1.4.0 at the moment, since it does not use the docker_container module. Will likely expand support once v2 is feature complete. [1] https://www.computest.nl/advisories/CT-2017-0109_Ansible.txt [2] https://lwn.net/Articles/711357/ * Better distribute role dependency documentation * Improved Molecule output of components * We should not skip important tests Removed the marker and updated the tests slightly. * Few small fixups with verbiage * Prefix files managed by molecule with a header * Prune the ephemeral directory before we begin * Added dependency functional tests Since we are no longer testing the explicit order of the dependency unit tests. A functional test should verify the subcommand is working as expected. * dependency molecule file cleanups Did not need multiple-instances or groups for this test. * Added verifier functional tests Since we are no longer testing the explicit order of the verifier unit tests. A functional test should verify the subcommand is working as expected. * Fixed up driver and host_group_vars * Nested docker functional tests under driver. * Implemented child tests in host_group_vars. * No longer manually add scenario name in host_vars The user no longer needs to manually add the scenario_name to the host_vars section of molecule.yml. * Correctly Prune the ephemeral directory We should prune before each subcommand. Previously, Molecule was pruning on Config init. * Moved yaml loading to functions * Entire config handling rework Config does not handle merging dicts any longer. We simply should load the molecule file which the config object is passed. As a side-effect the tests were reworked around config handling. Was never happy with the way I originally did it. * Brought base coverage back to 100% * Added interpolation functional test * Nested functional tests under driver name * Init add of vagrant module (#729) This vagrant module is baked inside Molecule, since a native Vagrant module does not exist. It writes two files (`Vagrantfile` and `vagrant.yml`) into the `molecule.config.ephemeral_directory`. These files are used by the module to provision the instances. Fixes: #720 * Only run functional tests when binary found Also, bumped YAPF version. * Able to share playbooks between scenarios The playbooks can now reference paths built by Molecule due to molecule exporting these as inventory vars. This enables playbook sharing between roles. * Backported #732 into v2 * Don't pass an empty list to sh.bake verifier User would receive a warning when no tests were found. Lets hide that silly warning. * Back out vagrant setup/destroy playbook sharing Need to dig into this issue further. * update testinfra backend for docker driver (#733) * First pass at pylint linting * Render Vagrantfile through jinja * Updates to run on travis * Added a travis file * Share the Dockerfile between scenarios * Ran yapf * Corrected vagrant playbook sharing * Provisioner Verifier Dependency Lint env variables Molecule now accepts environment variables which are sent to the provisioner, verifier, dependency, and lint modules. * Updated vagrant templates to use ubuntu/xenial64 The files generated by `molecule init` should use newer linux. Also, as part of this found a use case where a instance may not have the dependencies to initially run ansible. * Backported verbose flag from #736 * [v2] allow tests to run on python35 - add python35 targets to travis and tox - update tox to run linter for all python versions - update tests that use asssert_called_once to test against call_count instead (assert_called_once is only supported for python < 3 and >= 3.6) - update tox config to not fail if a target interpreter is missing * Backported verbose flag from #736 * Check instance state on converge Check if instance is already created when running converge. * Additional py 3.5 compat fixes * Small fix from rebase * YAPF fix * Updated tox to run linting with travis tox * Removed unused fixture and doc string * Switched to logger vs prints (#740) Logging is a more pythonic approach. The print methods feel somewhat gross as this project has evolved. * Corrected coverage * [v2]Fix lint environments in tox (#738) * Fix lint environments in tox * Allow setting host_vars for localhost - Do not append scenario name to hostname when setting host_vars in molecule.yml - add test coverage for localhost host_vars * Improve vagrant tests Vagrant tests fail from time to time. Especially when running the entire `tox` suite. * Moved the vagrant tests to debian/jesse. The box is smaller, and boots faster than xenial. * Updated vagrant module to clear the instance config on destroy. * YAPF formatting * Bumped vagrant driver coverage * Initial LXC driver support The Ansible LXC modules do not support unprivileged containers. To properly use this driver, one must prefix every molecule command with `sudo`. * Updated Ubuntu, CentOS, and Mac OS INSTALL.rst * Correct LXC functional tests All LXC functional tests now pass. * Added information about the Vagrant driver This information will be rendered into the configuration rst docs. * Added docstrings to logger * Moved install python for Ansible to converge pb Previously the vagrant functional test was not testing the init command via the vagrant driver. We cannot perform additional tasks at the setup phase, moved this to the converge phase. See the vagrant driver class docstring for further details. All tox tests now pass. * Updated raw trailing/leading newline Made these cleanups on the vagrant driver, ported these over to lxc and docker as well. * Initial LXD driver support test/functional/lxd/test_command.py ............... * Working through LXC testing * Found a few issues with incorrect install instructions. * Xenial LXC passes all functional tests. * Centos 7 LXC does not boot a container yet. 15 passed in 886.90 seconds * LXC CentOS passing functional tests 15 passed in 1854.07 seconds * No need sudo pip install in LXD Vagrant testing This was copy-pasta from LXC configuration, since LXC tests currently require sudo to work. Need to figure out unpriv containers in ansible. * Backported the Goss verifier into v2 Goss is now a first class and supported Molecule verifier. This is not the case in v1 Molecule. * Cleanup Dockerfile template a bit (#747) * SSoT around driver and verifier Provide a single source of truth for the verifier and driver. Managing this list by hand is unwanted. * Added converged/created state to `molecule list` * Provisioner cleanups * Prefixed these methods with the provisioner's name. * Removed inventory_file hard coding from testinfra. * Backport of #749 to v2 * Backported #770 to Molecule v2 * Corrected doc builds * Added Python 3, 3.5, and 3.6 to setup.cfg * Backported #784 to v2 * Backported #787 to v2 * Backported #792 to v2 * Corrected silly way of handling default dict vars * Backported #792 to v2 for Ans 2.3 * Backported #800 to v2 * Pinned requirements to newer versions * Updated anyconfig to current Anyconfig introduced quite a few breaking changes from 0.7. Took some time to port to current. * Revert "Backported #787 to v2" This reverts commit 035d024. * Bumped version of testinfra * No longer hard code testinfra with inventory path Now referencing a variable which points to Molecule's ansible inventory. Allows users to better organize their test files if they choose. * Removed OSX specific cleanup shutil.rmtree * Added 2.3 Ansible support to README * Removed try/except around destroy in tests Functional tests should not throw an exception when tearing down. Lets make sure we don't mask this. * Initial work of static driver This driver allows one to supply externally managed instances to molecule. Molecule skips the create/destroy playbooks on this driver. It is up to the user to properly configure ansible connect to these systems. It should be noted, one cannot connect externally to lxc, lxd, or docker when using their respectively named `ansible_connection`. The containers would need to be running ssh for this to be possible. `molecule test` runs successfully against local statically managed docker instances. Static Docker functional tests are passing as well. * Add ssh connection support to static driver The static ssh driver assumes the configuration will be performed in a ssh config file, and passed to ansible via the `ansible_ssh_extra_args` and `ansible_scp_extra_args` vars. Fixes: #778 * Added testing documentation * Backport of #821 to Molecule v2 * Cleaned up Testing and Contributing docs * Added scenario examples to documentation per #817 * Added a env property to config class A "global" env property will be used to pass global environment variables to all sh sub calls. This allows us to pass a consistent set of env vars throughout Molecule. * Print Molecule and Ansible environment on `--debug` * Set StrictHostKeyChecking to no Cleaned up drivers on how they deal with connection options. The vagrant driver was not working due to a machine switch. Also, cleaned up login_args to be clearer about use. * Reduce duplication in functional tests Still not perfect, need to rework static driver tests. * Corrected usage in lint docstring * Rename scenario to scenario_name in functional * Scenario and Provisioner rework / driver on CLI * Implemented passing driver on command line Ability to pass a driver on the command line to certain subcommands. Molecule will remember the last successful driver used, and continue to use the driver for all subsequent subcommands, until the instances are terminated. * Reworked scenarios and provisioner playbook configuration Scenarios should not have knowledge about the provisioners internals. Therefore scenarios referencing playbooks needed reworking. Moved existing functionality into the provisioner before further refactoring. * Support multi driver playbooks A role can be shipped with multi driver provisioning (setup/teardown) playbooks. Pairs nicely with the CLI changes mentioned above. * Yeah, OCD * Pass additional molecule settings to the env * Added a `--format` flag to `list` Part of adding this flag was to address functional tests always running destroy when not necessary. * Corrected a docstring note to important * Driver name rework and additional functional test * Rework driver name handling Made `driver.name` a setter which config sets once the driver has been determined. Provides consistent use throughout Molecule vs needing to call a method to determine driver name. * Added a functional tests for multi-driver Missed adding a functional test which uses the new `config['provisioner']['playbooks'][$driver] dict. * Add functional test logging Ability to turn on stdout/err logging in functional tests, by passing `-s` to pytest. * Restored CHANGELOG from v1 series * Corrected AUTHORS Took this opportunity to correct core contributors. These are contributors who have submitted substantial code and influence into the direction of Molecule. * Ported CI documentation over from v1 * Updated CHANGELOG with v2 changes As part of updating the changelog, added a porting guide, and corrected a bug with scenario init discovered while creating the porting guide. Corrected functional test to test for this condition. * Corrected remaining non-wrapped molecule calls * Added all group to generated inventory Added a functional test to cover this situation. * Added OpenStack driver This commit is much larger than it should be. I ended up touching things I probably should not have touched. This commit adds the OpenStack driver, and establishes a pattern others can follow when adding new drivers. Will use the lessons learned here to rework the Vagrant driver. * Converted vagrant driver to new pattern Using the new driver pattern in playbooks, that was hashed out while developing the OpenStack driver. * Move tests to new testinfra host fixture * Default scenario now default through CLI The only subcommands which do not default the scenario name are `test` and `list`. All other default to the default scenario. * Added _NAME to a few of the molecule env vars * Added molecule info to --help * Huge functional test refactor Finally able to understand parametrize. Reworked tests to better utilize this knowledge. Reduces a ton of duplication. * Added EC2 driver along with functional test fixes (#832) * Added EC2 driver Added the long awaited EC2 driver. Because of the way EC2 works, needed to perform playbook gymnastics in the setup/teardown playbooks. Specifically, b/c instances cannot be created or deleted by name. This required instance_ids to be persisted for use later. * Using `sh` magic flag parser. * Corrected some broken assumptions in functional tests. * Removed unnecessary duplicate testinfra tests. * Reworked skip_test to skip static tests (for now). * Removed (for now) static tests boostrap. * Uncoment and fix a functional test I made a booboo when testing the EC2 driver. Missed a functional test. Will submit a follow-up pull request to relocate the ssh-keypairs into an ephemeral directory wiped out after each scenario run. * Reloacted ssh_key into an ephemeral directory Prevents potentially valid keys from laying around the filesystem. * Better failure message when scenario not found Previously we would error stating molecule.yml cannot be found, when the scenario could not be found. * Better error when molecule.yml isn't found * Flake8 was running w/o molecule's environment * Reworked scenario name The scenario name cannot be configured through`molecule.yml`. It is determined by the directory name of the scenario. The config value references a `MOLECULE_SCENARIO_NAME` environment variable, which is correctly interpolated. However, it provided so that the config is symmetrical. It may be removed in the future, if the source of much confusion. * Added provisioner docs about provisioner playbooks * Added a FAW * Added information on autocomplete * Updated docs with --pre release install info * Revert "Reworked scenario name" This reverts commit 0c6a033. * Correct spacing in a couple scenarios * Rework scenario name part deux The scenario name cannot be configured through `molecule.yml`. It is determined by the directory name of the scenario. It provided so that the config is symmetrical. It may be removed in the future, if this becomes a source of confusion. * Correct role init usage * Chomp trailing spaces consistently * Updated dockerfile init file Contains molecule v1 RUN commands. * Ability to override the test directory * Small cookiecutter molecule.yml cleanups Trying to pass more data through molecule.yml instead of manually defining in create. Still some work to do around keypairs and security groups. * Namespaced molecule ansible plugins * Fixes discovered from functional tests * Add Ansible version check * Ported v1's development doc * Added cleanup task to development doc * Standardized formatting of ci doc * Pin pytest Running into a bug with pytest 3.1.0 and pytest-verbose-parametrize. Pinning for now. * Pin new version of pbr Was not compatible with newer Sphinx. * Define ansible ssh common args When running under tox the path is too long, which causes SSH failures. Defaulted to a shorter path. * Moved rdoc to base class * Reuse os_walk in command.base.prune() * Remove host/group vars before rewriting them * Corrected scenario functional tests yet again * Added base provisioner class This is potentially setting molecule up for future Helm provisioner. * Added dict_at_index filter plugin Having ansible convert a dict to a list isn't trivial inside a playbook. Much easier to do in python, so created a filter plugin. This also prevents the Docker create play from unnecessarily looping over each platform. > Most everything one hates with Ansible can be corrected with a filter > plugin. - Said None * Ability to init a role with cookiecutter template Users may wish to supply their own Molecule templates. Molecule uses cookiecutter internally, it makes sense to simply use templates provided by a cookiecutter repo. * Added current galaxy init meta/main.yml * Cleanup config module methods * Bumped testinfra deps for @tknerr * Ported v1 trailing whitespace linter to v2 This is not currently enabled. Not sure how I would like to approach this. * Switch to utility open functions * bumped for release of 1.25 (cherry picked from commit e2a43c5) * Test vagrant interface is created * Don't hard code platform_box in vagrant destroy.yml (#856) * Missed a scenario in #856 * Bumped yapf * Ran YAPF * Resynced goss library 7b92738 brought in a bug. * Brought in updated sh This version of sh corrects a py3 issues we ran into. * A few python 3.6 fixes Still cannot run python 3 against Ansible, due to Ansible not seeming to fully support py3. * Forward port idempotence stdout on error Fixes #835 * Added attribution to test case * Additional py3 fix Bug was introduced at a13d5a2. * Moved vagrant module examples into sphinx * docs: fixed initializing new scenario (#861) * Moved vagrant module examples into sphinx * Added additional Vagrant Molecule Module docs Along with documenting the molecule vagrant module, corrected a few bugs which were found. * Preserve special ANSIBLE_ envs with their value ANSIBLE_ like environment variables have higher precedence over their equivalents inside ansible.cfg. Thus, if the envs are set on CLI, the roles_path, library and filter_plugins inside ansible.cfg are completely ignored. No longer set these inside ansible.cfg, and pass them as env vars to the provisioner and ansible-lint. This prevents one from providing these on the command line, since molecule.yml provides a mechanism for setting env vars. Fixes: #845 * Lame attempt at unit/functional targeting Not really fond of this approach -- feels anti-tox. * Added v1 docs to the readme This will help clarify usage once v2 is merged to master.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A molecule v2 tracking issue.
Add environment variable handling to molecule.yml. There is already a v1 PR referencing this work #667.
Lets port this work when it lands to v2.
Upstream discussion.
Test cases and implementation to mimic.
The text was updated successfully, but these errors were encountered: