openstack playbooks

[joshmoore]

This removes the requirement on checking out a branch from @manics as well as the bootstrap script. This means the benefits described in the README are no longer possible, but with the introduction of the dynamic inventory I think this is balanced out. The general workflow is actually quite nice:

• ansible-playbook ... os-create.yml
• ansible-playbook ... os-omero.yml
• ... any other playbooks ...
• ... do your testing ...
• ansible-playbook ... os-delete.yml

I'd even like to see this runnable by devspace in a "ci" tenancy.

Questions:

• At this point should openstack/*.md moves to docs and openstack/examples/*.yml move to ansible?
• With 41ecc02, can 00_openstack.sh be reduced/removed?
• Remove omero usage from the os-*.yml files?

[mtbc]

directory

[manics]

omero_vm_flavour='My Keypair': change the param value to an example flavour (m1.* are standard)

[joshmoore]

...ok, but this is from you! 😄

[joshmoore]

Nope. I'm just wrong.

[manics]

Since you're editing this doc, worth mentioning that deleting the security group will fail (and can be ignored) if anything else is using that group?

[joshmoore]

Yeah, I looked into: "if unused" but didn't find anything. Will push.

[manics]

👍, a few small comments inline.

FYI I was thinking of an all-in-one playbook for the IDR to spin up one or more VMs (e.g. separate postgres, I think this is needed to avoid future performance/debugging issues), and to run the required playbook roles- I think it's possible with the add_host. But that's for a future PR

[manics]

One more, fix the playbook name at the at the top of os-delete.yml: # Delete VMs and security groups created by idr-os-playbook.yml

[joshmoore]

Pushed a first round of fixes. Need to discuss groups first.

[joshmoore]

Pushed better group names. Done unless you have suggestions.

[joshmoore]

FYI I was thinking of an all-in-one playbook for the IDR to spin up one or more VMs ... I think it's possible with the add_host. But that's for a future PR

Same here. I was assuming with an include an add_host wouldn't be necessary, but isn't so bad if it is.

[joshmoore]

add_host example in place, using files migrated from https://gist.github.com/manics/d37fef6dcb7d3f946fb177aedf79b3c2

[manics]

~centos/idr-metadata git checkout is owned by root, but I can't see why.

[manics]

Style fixes in joshmoore#1

[manics]

@joshmoore Are you planning anything else?

[joshmoore]

I don't have anything in flight. My top hope was to get the volumes auto-mounted but I failed to cleanly get the device name from os_server_volume (i.e. whether it already exists or not). We could go for a map of device names to volume names, but I haven't started on that.

[manics]

Another strange problem: limiting to -l localhost,FOO (so that it won't affect existing instances) doesn't work unless FOO already exists:

$ ANSIBLE_ROLES_PATH=../../ansible/roles ansible-playbook -i inventory uod-os-idr-omero.yml -vv -e omero_vm_name=FOO -e omero_vm_key_name=XXX --list-hosts -l localhost,FOO

No config file found; using defaults
ERROR! Specified --limit does not match any hosts

[joshmoore]

@manics : pushed my latest fixes and the proposed git mv structure from my description question.

[manics]

One nit-picky comment: You've got a mix of long (multiline) and short form YAML in some playbooks. If it's easier I can fix it in my pending commits.

[manics]

(Sorry, accidentally clicked close instead of comment)

[manics]

https://github.com/joshmoore/infrastructure/compare/os-playbooks...manics:pr70?expand=1

Main changes (you should be able to fast forward):

• There are now three separate VMs (Postgres, OMERO, gateway/caching proxy).
• Replaced omero_vm_name with vm_prefix which is used to prefix the names of all three VMs (-omero, -gateway, -database)
• I've reverted 8013083 for now, but you can limit hosts using ansible's -l parameter, e.g. -l vm_prefix-*, -l vm_prefix-omero.
• Added a wrapper script inventory/openstack-private.py. inventory/openstack.py only returns instances with floating IPs by default, you have to pass --private to see the private IPs which is necessary if you're running ansible via an ssh gateway host.
• Modified postgresql role to allow non-local connections

Still more to come

[manics]

Forgot to say so far I've mostly been testing with uod-os-idr-omero.yml. os-create.yml contains add_hosts tasks which means the openstack dynamic inventory isn't strictly needed if you're setting up the whole system from scratch, however it is required if you want to run os-omero.yml separately. Also be aware that os-create.yml automatically sets the ssh proxy variables to go via the gateway if single_floating_ip: True which is essentially what we have on Embassy. Change this to False to return to the previous behaviour of one floating IP and direct ssh connections for every instance. If you're just running os-omero.yml you'll need to configure those ssh proxy variables somewhere else.

E.g. ansible-playbook -i localhost uod-os-idr-omero.yml -e vm_prefix=test -e vm_key_name='"key name"'

[joshmoore]

Merging this as is since things are getting complicated. Leaving @manics to figure out the next phase.

[manics]

Refactored my non-openstack changes into two separate PRs:

• Nginx proxy state typo #74
• Allow external connections to postgresql server #75