Prepare the modules for inclusion in Ansible upstream - ansible collection

[OndrejHome]

Idea:
Prepare modules for inclusion into Ansible upstream as group of related modules for work with pacemaker cluster.

https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_in_groups.html

This Issue is used to track what is needed for inclusion and also for sharing the information on the progress of preparation for inclusion that already several people asked me for. Once this has a bit better shape you may find here also information on how to help with this. Stay tuned for more.

Update 2021/06: Consider making cluster related libraries and roles into Ansible collection.

[OndrejHome]

ansible-test sanity state as of commit a3cfc4e :

# for i in pcs_{auth,cluster,constraint_colocation,constraint_location,constraint_order,property,resource_defaults,resource}; do echo "===== $i"; ansible-test sanity -v --tox --python 3.6 --skip-test symlinks $i|grep ERROR|grep 'lib/ansible'; done
===== pcs_auth
ERROR: Command "tox -c test/runner/tox.ini -e py36 -- /usr/bin/env LC_ALL=en_US.UTF-8 /root/ansible/bin/ansible-test sanity -v --python 3.6 --skip-test symlinks pcs_auth --metadata metadata-9kU74z.json --truncate 159 --color no --requirements" returned exit status 1.
ERROR: lib/ansible/modules/clustering/pcs_auth.py:140:13: E123 closing bracket does not match indentation of opening bracket's line
===== pcs_cluster
===== pcs_constraint_colocation
ERROR: Command "tox -c test/runner/tox.ini -e py36 -- /usr/bin/env LC_ALL=en_US.UTF-8 /root/ansible/bin/ansible-test sanity -v --python 3.6 --skip-test symlinks pcs_constraint_colocation --metadata metadata-QIvJ3s.json --truncate 159 --color no --requirements" returned exit status 1.
ERROR: lib/ansible/modules/clustering/pcs_constraint_colocation.py:175:161: E501 line too long (172 > 160 characters)
===== pcs_constraint_location
===== pcs_constraint_order
ERROR: Command "tox -c test/runner/tox.ini -e py36 -- /usr/bin/env LC_ALL=en_US.UTF-8 /root/ansible/bin/ansible-test sanity -v --python 3.6 --skip-test symlinks pcs_constraint_order --metadata metadata-cGYVEF.json --truncate 159 --color no --requirements" returned exit status 1.
ERROR: lib/ansible/modules/clustering/pcs_constraint_order.py:187:161: E501 line too long (191 > 160 characters)
ERROR: lib/ansible/modules/clustering/pcs_constraint_order.py:219:161: E501 line too long (161 > 160 characters)
===== pcs_property
===== pcs_resource_defaults
===== pcs_resource
ERROR: Command "tox -c test/runner/tox.ini -e py36 -- /usr/bin/env LC_ALL=en_US.UTF-8 /root/ansible/bin/ansible-test sanity -v --python 3.6 --skip-test symlinks pcs_resource --metadata metadata-AcFaMJ.json --truncate 159 --color no --requirements" returned exit status 1.
ERROR: lib/ansible/modules/clustering/pcs_resource.py:337:161: E501 line too long (317 > 160 characters)
ERROR: lib/ansible/modules/clustering/pcs_resource.py:339:161: E501 line too long (198 > 160 characters)
ERROR: lib/ansible/modules/clustering/pcs_resource.py:341:161: E501 line too long (179 > 160 characters)
ERROR: lib/ansible/modules/clustering/pcs_resource.py:379:161: E501 line too long (184 > 160 characters)

Mostly just long-lines.

Once resolved the testing for travis should be improved to include above testing.

Currently targeted platforms for inclusion to upstream would be for CentOS/RHEL 7 and Fedora 29, which means support for pcs-0.9 and pcs-0.10.

[tleguern]

Hello, 2018 was a long time ago considering Ansible progress. Do you plan on converting your roles into a collection? I think it is particularly suitable :)

[OndrejHome]

Yes, this was mostly to provide also some one else asking me with some link where progress can be tracked (if it makes any progress). I got stuck mostly on part where for inclusion the 'tests' are needed - which are done by hand so far.

Making the modules and related roles into collection is however and interesting idea. I will give it a thought, but no promises on when this would happen as my summer looks so far busier than usual.

[YannickEvers]

Hello,
is the Collection still planned? That would be super helpful for us, because we would like to use the modules in a role inside our Collection. And there, unfortunately, you can only specify other collections as a dependency.

[OndrejHome]

Hello and Thank you for interest and continued interest in getting this into collection.

1. This was for quite a time on low priority for me as the roles were good enough to just use the modules from them, therefore If possible could you please let me know more exact use case where the current approach of just using role is not good enough?
   (I can imagine that probably something like AWX/tower might be challenging, but I'm not so actively using those to be able to tell. If you have some example of "I need to do a,b,c,d now, but if this was collection I could do just e,f and be done" that would give me a good idea on why to reconsider priority of this)

2. I'm considering for initial collection test the mix of 'pcs-modules-2' and 'ha-cluster-pacemaker' roles to have both modules and sample role that gets the whole pacemaker cluster up and running. Would be someone able to give it a test once such collection is available? (my current estimate for such testing collection is by the end of September or early October) - simple 'yes' or 'thumbs up' is sufficient.

As I'm not sure how to put whole collection together (yet) I would like to keep the history of individual roles and modules as they are. That is why I expect that this experimentation could take me some time to get right. Any good example of project going from several repositories for roles into single collection keeping the history is greatly welcomed.
At this moment I'm in acute need for summer vacation due to come soon, therefore I expect to start exploring some approaches from September.

[OndrejHome]

Hello for those following on slowly going stuff :)

Long story short: Initial test of collection is at https://galaxy.ansible.com/ui/repo/published/ondrejhome/ha_cluster/ available for testing - note that things are not polished and "collection-style" documentation is not there yet. More complete collection may appear during this October or early November. Do NOT depend on the locations of modules within collection yet.

---

Initial preview of how the collection may look like here - https://galaxy.ansible.com/ui/repo/published/ondrejhome/ha_cluster/
NOTE: the version is intentionally very low and the changes are so far only uploaded via collection tarball - the git repository is still in preparation for publishing - the main challenge being the merge of several repositories with history while preserving it.

From the play with collections so far I can see following challenges/things that needs to be done:

• need to rename some variables (HALVMtype in roles/lvm being the most significant)
• converting all modules to FQCN - for now only the ones that are not build-in were changed
• reviewing the READMEs for roles and making a summarizing the content in the main README file of collection
• creating directory with example playbooks that are mentioned in roles so they could be used directly in AWX

For now the above will be ongoing in local git repo and later published as (NEW) separate git repository. For some time the content in it will be functionally same as the other git repositories from which it got the initial data, but at some point that may change. I don't see any sane way of having the roles and modules in separate git repositories as they are now and just putting those together into collection without need to make both roles and modules "collection aware" (for roles there is some new attribute saying they are standalone that indicates which type of role it is, for modules the issue is having them coming from several git repositories and being put together in one directory in collection (git submodule cant be used to combine stuff in same directory, while ansible ignores symlinked modules). Also as very recently the galaxy.ansible.com switched to galaxy_ng, there are way too many things that stopped working for me to call it smooth transition (many links just redirects to homepage, previously uploaded roles links are different, web UI doesn't show which platforms(OSes) the roles are intended for).

---

If you have any feedback on collection in preparation, feel free to comment in here.

[OndrejHome]

Hello everyone,
Thank you for all those curious and trying the initial versions (0.0.1~0.0.5) - now deleted.
Public git repository with collection and first version of collection (1.0.0) are now available!

• Ansible galaxy collection (ondrejhome.ha-cluster): https://galaxy.ansible.com/ui/repo/published/ondrejhome/ha_cluster/
• Github repo (ansible_collection.ha_cluster): https://github.com/OndrejHome/ansible_collection.ha_cluster

If you encounter any issues with collection please direct the found issues into https://github.com/OndrejHome/ansible_collection.ha_cluster/issues or use the email ondrej-xa2iel8u@famera.cz for your reports. Please make sure that you use collection version 1.0.0 or newer for your reports.

The new (collection) repository is combination of previously used (roles) repositories and plan (for now) is to primarily accept changes to original (roles) repositories and reflect changes to the one with collection (with exception of changes that makes sense only for collection itself).

This repository and other ones included in collection will be gradually updated with information about existence of collection, but still maintained for stand alone use.

--
Ondrej (온드레이)