A Yeoman Generator for Living Atlas Ansible Inventories
- Rerunning the generator
- Ansible Wrapper
- Any problem with this generator?
- But why this?
- Can these quick-start inventories solve all this?
- About Yeoman
This is a simple yeoman generator, so, just by asking some simple questions about your Living Atlases Node, you will get a functional configuration for the setup of your Virtual Machines (VMs) using
The main goal of this utility is to allow you to configure, install & maintain a Living Atlas infrastructure in a fast & smooth way.
In others words, with the
ansible inventories produced by this
yeoman generator and the ala-install
ansible playbooks you should deploy the main LA services quickly and without too much pain.
Also, thanks to
yeoman, you can rerun this generator, for instance, to enable new modules in the future, or to enable eg. SSL, etc. and reuse your previous settings and compare the differences (see the screenshots).
npm install -g yo
Wait!, as this
generator-living-atlas is still somewhat experimental, it's not published in
npm. So if you want to test it, you can clone this repo and just use it like:
cd this-cloned-repo npm link cd /tmp # or other directory yo living-atlas
As this is a
yeoman development environment you can get some
EACCESS errors, so check the yeoman setup guide for proper configuration.
If we decide to publish this generator in the future, the normal use will be like:
npm install -g yo npm install -g generator-living-atlas
Then generate your new project:
yo living-atlas --replayto reuse all the previous responses and regenerate the inventories with some new modifications (or improvements if you update this repo). More info below.
yo living-atlas --replay-dont-askif you only want to repeat the inventories generation (for instance, with a new version of this generator to get some update, or when you edit carefully the
../.yo-rc.jsonanswers file to, for instance, enable ssl or some service, and only want to regenerate the inventories with the changes).
- Also, you can use
--debugto see some verbose debug info.
A screen recording re-running the generator but using another solr hostname and switching to use SSL (see the differences in the generated inventories):
After running the inventories following the instructions from the generated README over two VMs (like in the Paris 2019 Workshop):
Using it for Tanzania demo session:
Differences beween two runs:
We can use these inventories as a base for extracting documentation via
doxygen (or similar):
Rerunning the generator
You can rerun the generator with the option
yo living-atlas --replay to reuse all the previous responses and regenerate the inventories with some modifications (if for instance you want to add a new service, or use a new version of this generator with improvements).
--replay-dont-ask you can regenerate the playbooks with previous answers without being asked again. This is useful for new versions of this generator, or if you manually edit
.yo-rc.json where your answers are stored, or for automated processes (like ci integrations).
We recommend to override and set variables adding then to
quick-start-spatial-local-extras.yml without modification of the generated
quick-start-spatial-inventory.yml, so you can rerun the generator in the future without losing local changes. The
*-local-extras.sample files will be updated with future versions of this generator, so you can compare from time to time these samples with your
*-local-extras.yml files to add new vars, etc.
Furthermore the utility
ansiblew is generated.
ansiblew is an
ansible-playbook wrapper that can help you to use the inventories and can be easily modified by you to suit your needs. It depends on
python-docopt package (
sudo apt install python-docopt in debian and derivatives).
$ ./ansiblew --help This is an ansible wrapper to help you to exec the different playbooks with your inventories. By default don't exec anything only show the commands. With --nodryrun you can exec the real commands. With 'main' only operates over your main host. Usage: ansiblew --alainstall=<dir_of_ala_install_repo> [options] [ main | collectory | ala_hub | biocache_service | ala_bie | bie_index | images | lists | regions | logger | solr | cas | biocache_backend | biocache_cli | spatial | all ] ansiblew -h | --help ansiblew -v | --version Options: --nodryrun Exec the ansible-playbook commands -p --properties Only update properties -l --limit=<hosts> Limit to some inventories hosts -s --skip=<tags> Skip tags -h --help Show help options. -d --debug Show debug info. -v --version Show ansiblew version. ---- ansiblew 0.1.0 Copyright (C) 2019 living-atlases.gbif.org Apache 2.0 License
So you can install the CAS service or the spatial service with simple commands like:
./ansiblew --alainstall=../ala-install cas --nodryrun
./ansiblew --alainstall=../ala-install spatial --nodryrun
or all the services with something like:
./ansiblew --alainstall=../ala-install all --nodryrun
--nodryrun you will get only the full
ansible-playbook command to use these inventories .
- Add basic services (
- Add domain/context and service subdomains support
- Disable caches when using the same host for
- Improve README hosts
- Improve README playbook commands
- Add inventories to add extra vars without modify the generated inventories
- Store = true if you running it for the first time
- Add other administration info (technical_contact, orgEmail, etc)
- Recommend to use git to track your inventories changes
apikey_db_dumplocation relative to
sslself signed certs support when ssl enabled
- Improve generator for only a domain and
local-extras.samplefiles that can be updated in the future without lost local changes in
- Generation of passwords on first run and increase security by default
- Added option
--replay-dont-askto regenerate the playbooks with previous answers without asking again
- Improble BIE/biocache etc descriptions (records, species)
- Demo index.html uses
biocache_hub_context_pathetc that does not works with multi hosts
collectory_app = ala-collectoryin main playbook
biocache-dbinstead of missing playbook
- Clean nginx fragments for each hostname
- Update to use the new image-service (grails3)
- Use a different playbook than demo that do not install by default all main services so we can choose a different host for them
--tagsto ansible wrapper
- Generate CAS salt
- Improve questions when not using subdomains using and /context instead
- Added dashboard support
- Better testing
- Currently running several services in the same machine and the same domain doesn't configure correctly
vhosts. See this enhancement issue in
ala-install. Workaround: Use different hostnames per each service in the same machine. So, if for instance, bie and bie-ws are in the same machine, this machine receives multiple names (like bie.example.org and bie-ws.example.org) and
nginxdon't overwrite each service vhost configuration.
Any problem with this generator?
Please add an issue give us details of your problem and we'll try to help you and to improve the generator. Thanks!
But why this?
Ansible is a fantastic tool for manage LA node infrastructures. But nowadays we have some problems with our current
ala-install inventories/roles & documentation that make their use difficult by newcomer nodes, but also for other non-Australian nodes' maintenance:
- There are many config properties that are not documented and this makes it quite difficult to tune a LA node
- Sometimes the
/data/*/config/*propertiesare well commented but not their source inventories
- From time to time new properties appear but other non-ALA nodes don’t notice
- We end up using several self-made inventories, duplicate properties (like orgName, urls, etc), and many times this is a source of problems (like code duplication). Some minor sample:
- There is a lack of more real production-ready inventories (you have to ask for them).
ala-installhas a structure that sometimes is difficult to find some samples or how to start to deploy some service (think of CAS)
- Many times you have to check the
config/*propertiesfor non configured or default properties, search the
ansiblerole code to know how to configure these variables, re-run
ansiblewith these new variables and re-check the
- The demo inventory (or the last Paris workshop inventories) are a good sample but not enough for a new production LA node (lack of inventory variables documentation, missing important services like CAS)
- We have a lack of info about services' versions compatibility
- Sometimes default module versions in inventories or
LATESTnexus packages are not the correct/latest ones.
- It would be nice to choose strategies that are also usable by the ALA (not just LA nodes) - this might help maintenance.
Can these quick-start inventories solve all this?
No. This is only a Proof of Concept that tries to mitigate some of the previous problems.
If we think that some tool like this can be useful, we can add more functionality like:
- Better comments of properties in generated inventories for easy operation & tuning
CAS deploymentdone ✓
- Focus on having well maintained generators and their inventories
- Autogenerate documentation from the inventory comments with tools like
or maybe this experiment suggests to us another path to follow.
Apache-2.0 © Living Atlases