Use ansible os_tasks to create OOI service and endpoint. #6

gwarf · 2018-03-31T18:35:54Z

Use ansible os_tasks to create OOI service and endpoint.
Reorganize files.
Update ansible, travis and yamllint configuration.
Address complaints from yamllint and ansible-lint.

No more use OpenStack CLI from remote server. Group tasks in blocks. Shorten some long lines.

…ack client

brucellino

I suggest a few changes, overall.

Stick to the var: value style.
Avoid use of shell module in favour of mysql modules.

There are a few other cosmetic and style suggestions which can wait for future releases.

brucellino · 2018-04-01T10:23:42Z

playbooks/keystone_voms.yaml

@@ -6,7 +6,7 @@
    # XXX This should be configurable
    - name: Add default VOs LSC files
      copy:
-        src: ./lsc/
+        src: ./templates/lsc/


not to be a pedant, but "copy" takes static files. These should be taken from files/ instead of templates/

I can't find where this is written in the documentation, so perhaps I am remembering incorrectly.

brucellino · 2018-04-01T10:26:15Z

playbooks/igtf.yaml

@@ -1,5 +1,5 @@
 ---
- hosts: all
+- hosts: packstack
  become: yes
  become_method: sudo
  tasks:


In a later version, I would suggest re-using a role for this - AAROC.certificates or something from egi-qc

Yes we need to have a dedicated role for this, for sure.
@brucellino can you please open an issue for this?

Ok, done in #8

brucellino · 2018-04-01T10:27:23Z

playbooks/keystone_voms.yaml

+        - name: Add VOMS filter
+          blockinfile:
+            path: /etc/keystone/keystone-paste.ini
+            block: |


Shouldn't there be a regexp to say where this block should go ? It seems like this may become non-idempotent.

It's an inifile and this block is indepedant, so this shouldn't be that much a problem, but I will replace it by some inifile tasks, it will definitely be cleaner/safer.

brucellino · 2018-04-01T10:29:43Z

playbooks/ooi.yaml

+        gpgcheck: no
+        gpgkey: http://repository.egi.eu/community/keys/APPDBCOMM-DEB-PGP-KEY.asc
+
+    - name: Install OOI


maintain a common syntax for tasks. instead of one-liner (var=value) you should have one value per line : (var: value)
So instead of yum: name=python-ooi state=installed,

yum: name: python-ooi state: instaled

This format also makes it easier to catch what has changed when you do a diff (changes are only one per line)

brucellino · 2018-04-01T10:31:21Z

playbooks/ooi.yaml

+            section: composite:occi_api_11
+            option: keystone
+            # yamllint disable-line rule:line-length
+            value: cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit osprofiler authtoken keystonecontext legacy_v2_compatible occi osapi_compute_app_v21


This is quite a long line ! I would keep this in a var file, so that you can manage it easier, and use {{ keystone_vars }} or something like that. Maybe keep this for when this becomes a role.

Yes, I tried to also use the syntax with a > to allow to split the line, don't know exactly why but the task was changing/reloading all the time.
The fact is also that this line could change on different deployment, it's based on another value of the inifile. Need to find later a better way to manage this.

can't you just do a with_items: keystone_vars ? then you would see which one specifically changed

I think the problem was more a bug of ini_file with values split using > so not much we can address. As I said this shouldn't really be a static line, let's plan to update this in the roles and keep it like for the time being.

brucellino · 2018-04-01T10:36:39Z

playbooks/packstack.yaml

@@ -26,11 +29,7 @@

    - name: Install OpenStack {{ os_release }} repository
      yum: name=centos-release-openstack-{{ os_release }} state=installed


Use the var: value style
Make sure to keep the values using variables ( {{ }} ) protected by interveted commas :
name=centos-release-openstack-{{ os_release }} -> name: "centos-release-openstack-{{ os_release }}"

brucellino · 2018-04-01T10:38:19Z

playbooks/packstack.yaml

@@ -287,32 +286,59 @@
    - name: Fix endpoints in keystone database
      block:
        - name: Look for HTTPS 5000/v2.0 url
-          shell: mysql keystone -s -N -e 'select url from endpoint where url LIKE "http://%5000/v2.0"' | wc -l
+          shell: >


AVOID SHELL ! IT IS A WEAK MAN's CRUTCH !
Use something from http://docs.ansible.com/ansible/latest/modules/list_of_database_modules.html#mysql

I do not see a way with this modules to easily interact with the database apart using mysl_db with state: import and a .sql file.
I think that in fact it would be better to use the native os_ modules to edit the endpoints, could you please add a bug for this, as we will have to clarify what all the endpoints could be?

brucellino · 2018-04-01T10:39:03Z

playbooks/packstack.yaml

+            mysql keystone
+            -e "update endpoint set
+            url='https://{{ server_fqdn }}:35357/v3' where
+            url='http://{{hostvars[inventory_hostname]['ansible_default_ipv4']['address']}}:35357/v3';"
          when: https_35357_v3.stdout != '0'


try to use something from http://docs.ansible.com/ansible/latest/modules/list_of_database_modules.html#mysql for this block

brucellino · 2018-04-01T10:39:33Z

playbooks/packstack.yaml

@@ -388,3 +414,8 @@

    - name: restart neutron
      service: name=neutron-server state=restarted
+


use the var: value style

brucellino · 2018-04-01T10:40:35Z

.travis.yml

@@ -15,12 +15,15 @@ python:

 install:
  - pip install ansible
+  # - pip install shade shade-ansible


since there's a requirements now, why not simplify install by using it.

For the time being not everything is needed for the tests and it will speed things to not install shade that has a lot of dependencies.
To be changed once more through tests will be in place.

brucellino

What a beautiful review.

gwarf added 28 commits March 31, 2018 20:16

yamllint: be more tolerant on long lines

d470fa9

travis: install required python modules

5b4e47d

yamllint: fix conf for long lines

b384940

travis: simplify install and update yamllint call.

a77d6f6

yamllint: allow long lines

b15bf3e

ooi: use ansible os_ tasks to create OOI service and endpoint.

5a53689

No more use OpenStack CLI from remote server. Group tasks in blocks. Shorten some long lines.

travis: use ansible-lint

8ee500c

packstack: address yamllint complaints

3679765

ansible: do not gather facts by default

528a624

Address ansible-lint errors

3899eda

ansible: disable .retry files

3b7e470

Reorganize files

ec403b6

ansible: disable cowsay integration

7c6bb5a

ansible: improve ssh connection, enable callbacks

641a5bb

ansible: use inventory and set default user

c90d82e

README: simplify ansible CLI

0eb9909

igtf: update host group

83405c2

README: document usage of ~/.config/openstack/clouds.yaml with OpenSt…

ca4683f

…ack client

clouds: add yaml header

6effc43

requirements: install OpenStack client

bf1cceb

README: simplify exmaple usage

9310610

weapons: pubkey auth is a requirement

5a8b80b

projects: fix task names

11bb326

Reorganize files

3fe45fb

ooi: fix persistent refresh, disable useless backups

69bad25

ooi: disable line lenght check by yamllint

d6d7a34

travis: disable shade install as not yet used for testing

f09029e

travis: disable shade install as not yet used for testing, bis

4901728

brucellino suggested changes Apr 1, 2018

View reviewed changes

Playbooks: use same yaml formant/syntax everywhere

6d3e2e8

gwarf added 4 commits April 3, 2018 09:37

weapons: use package instead of yum

9534f05

Move statif files to a dedicated dir.

c0914ec

keystone_voms: use ini_file instead of blockinfile

ac66d76

pacstack: quote * in values

696ff0e

brucellino mentioned this pull request Apr 3, 2018

Improve tasks to make OpenStack configuration idempotent (no shell) #9

Open

brucellino approved these changes Apr 3, 2018

View reviewed changes

brucellino merged commit 541be5c into EGI-Federation:master Apr 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use ansible os_tasks to create OOI service and endpoint. #6

Use ansible os_tasks to create OOI service and endpoint. #6

gwarf commented Mar 31, 2018 •

edited

Loading

brucellino left a comment

brucellino Apr 1, 2018

gwarf Apr 3, 2018

brucellino Apr 1, 2018

gwarf Apr 3, 2018 •

edited

Loading

brucellino Apr 3, 2018

brucellino Apr 1, 2018

gwarf Apr 3, 2018

brucellino Apr 1, 2018

gwarf Apr 3, 2018

brucellino Apr 1, 2018

gwarf Apr 3, 2018

brucellino Apr 3, 2018

gwarf Apr 3, 2018

brucellino Apr 1, 2018

brucellino Apr 1, 2018

gwarf Apr 3, 2018 •

edited

Loading

brucellino Apr 1, 2018

brucellino Apr 1, 2018

brucellino Apr 1, 2018

gwarf Apr 3, 2018

brucellino left a comment

		@@ -26,11 +29,7 @@

		- name: Install OpenStack {{ os_release }} repository
		yum: name=centos-release-openstack-{{ os_release }} state=installed

		@@ -388,3 +414,8 @@

		- name: restart neutron
		service: name=neutron-server state=restarted

Use ansible os_tasks to create OOI service and endpoint. #6

Use ansible os_tasks to create OOI service and endpoint. #6

Conversation

gwarf commented Mar 31, 2018 • edited Loading

brucellino left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gwarf Apr 3, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gwarf Apr 3, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brucellino left a comment

Choose a reason for hiding this comment

gwarf commented Mar 31, 2018 •

edited

Loading

gwarf Apr 3, 2018 •

edited

Loading

gwarf Apr 3, 2018 •

edited

Loading