Updating to use deb packages #42

johandahlberg · 2016-04-01T14:54:39Z

I've update the roles and playbook to use the new deb-packages to install st2. Essentially I've converted the install guide to a playbook.

Sorry for the massive changes. I've tried as much as possible to dived this into commits that make sense, but I might have mixed things up in some places. I can now build this in the supplied Vagrant box, and I've also added a small smoke test role, which should confirm that some basic things are working as expected.

There are probably some things which still need fixing, especially in relation to making sure that all customization relating to the variables are working, but this is at least a working draft.

manasdk · 2016-04-01T15:15:33Z

This is awesome!

johandahlberg · 2016-04-01T16:04:07Z

Thanks. Don't know why Circle CI is failing though...

manasdk · 2016-04-01T17:51:14Z

@johandahlberg It failed on https://github.com/StackStorm/ansible-st2/blob/master/circle.yml#L28

TASK [rabbitmq : Install rabbitmq packages] ************************************
task path: /home/ubuntu/ansible-st2/roles/rabbitmq/tasks/main.yml:2
failed: [localhost] => (item=[u'rabbitmq-server']) => {"cache_update_time": 1459522939, "cache_updated": true, "failed": true, "item": ["rabbitmq-server"], "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"   install 'rabbitmq-server'' failed: invoke-rc.d: initscript rabbitmq-server, action \"start\" failed.\ndpkg: error processing rabbitmq-server (--configure):\n subprocess installed post-installation script returned error exit status 1\nErrors were encountered while processing:\n rabbitmq-server\nE: Sub-process /usr/bin/dpkg returned an error code (1)\n", "stderr": "invoke-rc.d: initscript rabbitmq-server, action \"start\" failed.\ndpkg: error processing rabbitmq-server (--configure):\n subprocess installed post-installation script returned error exit status 1\nErrors were encountered while processing:\n rabbitmq-server\nE: Sub-process /usr/bin/dpkg returned an error code (1)\n", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following NEW packages will be installed:\n  rabbitmq-server\n0 upgraded, 1 newly installed, 0 to remove and 1342 not upgraded.\nNeed to get 3909 kB of archives.\nAfter this operation, 4867 kB of additional disk space will be used.\nGet:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/main rabbitmq-server all 3.2.4-1 [3909 kB]\nFetched 3909 kB in 1s (2817 kB/s)\nSelecting previously unselected package rabbitmq-server.\n(Reading database ... 635558 files and directories currently installed.)\nUnpacking rabbitmq-server (from .../rabbitmq-server_3.2.4-1_all.deb) ...\nProcessing triggers for man-db ...\nProcessing triggers for ureadahead ...\nSetting up rabbitmq-server (3.2.4-1) ...\nInstalling new version of config file /etc/init.d/rabbitmq-server ...\n * Starting message broker rabbitmq-server\n * FAILED - check /var/log/rabbitmq/startup_\\{log, _err\\}\n   ...fail!\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following NEW packages will be installed:", "  rabbitmq-server", "0 upgraded, 1 newly installed, 0 to remove and 1342 not upgraded.", "Need to get 3909 kB of archives.", "After this operation, 4867 kB of additional disk space will be used.", "Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/main rabbitmq-server all 3.2.4-1 [3909 kB]", "Fetched 3909 kB in 1s (2817 kB/s)", "Selecting previously unselected package rabbitmq-server.", "(Reading database ... 635558 files and directories currently installed.)", "Unpacking rabbitmq-server (from .../rabbitmq-server_3.2.4-1_all.deb) ...", "Processing triggers for man-db ...", "Processing triggers for ureadahead ...", "Setting up rabbitmq-server (3.2.4-1) ...", "Installing new version of config file /etc/init.d/rabbitmq-server ...", " * Starting message broker rabbitmq-server", " * FAILED - check /var/log/rabbitmq/startup_\\{log, _err\\}", "   ...fail!"]}

If you login you might be able to re-run with debug ssh and can ssh to the container to see why that failure happens.

It seems that the Circle CI still has old rabbit-mq configurations files lying around. Hopefully this will fix that.

johandahlberg · 2016-04-04T09:02:48Z

@manasdk I've been investigating this a bit closer now. One thing that I noticed now is that Circle CI is using Ubuntu 12.04 as the base image, while the install instructions for Ubuntu / Debian states that Ubuntu 14.04 is supported. When trying this our in my own repo, triggering Circle CI everything worked when I updated the configuration to build on a Ubuntu 14.04 box. Would it be possible to make those changes the configuration of the main repo as well? This should also make the builds consistent with the supplied Vagrant file.

arm4b · 2016-04-04T10:47:03Z

Changed to Ubuntu 14 in Circle settings:

arm4b · 2016-04-04T10:51:00Z

roles/st2-web/tasks/main.yml

+- name: Install exact st2-web package
+  sudo: yes
+  apt:
+    name: st2web={{ st2_version }}.{{ st2_revision }}


st2web has independent revision number.
Eg. revision number for st2 could be 150, while for st2web latest revision number could be 6.

enykeev · 2016-04-04T11:09:16Z

The change itself looks logically sound to me and I can confirm the playbook produces working copy of st2 (including mistral and nginx deployment of st2web, but no chatops).

What I would like you to do is to clean up the code a little bit: remove extra empty lines, readme.md files in roles (or fill them appropriately), empty files.

I'm also kind of worried that name of the company in role meta may raise questions. I'm all for attribution, but since I'm myself struggling to figure out who we are now, we should probably escalate it to @dzimine.

johandahlberg · 2016-04-04T12:06:16Z

@armab Thanks for the comments. I'll make the changes you suggest.

@enykeev I've not added that chatops things since we're not using that part of the StackStorm functionality around here. I can for sure clean things up a bit. When it comes to the readme's do you have any template for that I could have a look at to see what to include?

I have no problem changing meta files - or removing them completely for the time being (since I guess they are only really necessary when publishing to galaxy). Do you have any contribution guidelines somewhere detailing what to do with those things?

enykeev · 2016-04-05T04:43:28Z

@johandahlberg I don't think we have any kind of guidelines or templates for such things. Deleting meta and readme for now seem like a good idea, though.

arm4b · 2016-05-02T12:34:23Z

roles/mistral/defaults/main.yml

@@ -1,4 +1,4 @@
-mistral_version: 0.13
+mistral_version: 1.3.2-99


1.4.0 is latest mistral stable version from https://packagecloud.io/StackStorm/stable. Let's stick to some revision from that version.

On related note, I would prefer to avoid such pinning in default config values. So if version is not explicitly set, playbook installs latest stable by default (with accordance with st2 version).

Anyway, it's something to rethink later. For now it's good enough.

I guess you mean something like what's used here: https://github.com/johandahlberg/ansible-st2/blob/use_deb_packages/roles/st2/tasks/main.yml? If so I can fix it now, shouldn't be to tricky.

Yeah, something like that would be definitely better.

johandahlberg · 2016-05-02T13:36:40Z

@armab Thank you for all the great comments. I've addressed them now and pushed the changes.

arm4b · 2016-05-02T14:33:53Z

Another very minor issue: can you also delete that big WARNING (first lines) from the README.md as part of your PR?

johandahlberg · 2016-05-02T14:39:38Z

Not a problem. 😃

arm4b · 2016-05-02T14:41:53Z

Everything else looks good for me 👍
Thank you very much @johandahlberg for your hard work and huge amount of changes 💯

Just a notice, probably (if I'll maintain this repo) there will be more refactorings with major changes in logic. Hope that's OK.

For now @enykeev will come back later and review changes once again & merge.

johandahlberg · 2016-05-02T14:47:14Z

@armab No problem at all. I'm just happy to be able to contribute, and that once this is merged there it makes it it easier for us to deploy the latest st2 version around here. 😄

enykeev · 2016-05-05T16:40:08Z

roles/st2/handlers/main.yml

-  with_items: st2_services
+  command: st2ctl restart 
+
+- name: restart st2api


We need to also restart st2stream along with st2api for pretty much every case. See https://github.com/StackStorm/st2-packages/pull/293/files. If you don't do that, attempt to connect to stream with token will result in an error, something along the lines 'unexpected query parameter x-auth-token'.

And while you at it, I'd also ask you to change st2 restart command to reflect https://github.com/StackStorm/st2-packages/pull/285/files

enykeev · 2016-05-05T16:47:36Z

The rest looks good to me 👍

johandahlberg · 2016-05-06T10:05:35Z

Thanks for the feedback @enykeev. I've made sure the st2stream and st2api restart together, and that a restart of st2 also triggers a reload.

Synchronize docs with recent #42 which installs new packages

Docs cleanup for #42

arm4b · 2016-05-06T14:34:43Z

Merged,
Thanks guys! 👍

Johan Dahlberg added 16 commits March 31, 2016 10:14

Only install mongodb-server package

8f7471d

Install rabbitmq-server from ubuntu repos

7b48d88

Adding role for st2-repos

0539c96

Changes to mistral setup

ff53402

Removing non-used mistral role files

843b652

Notify mistral restart

8b873d8

Install st2 and enable auth

f325d59

Remove unused st2 files

9740bd0

Add st2-repos as dependency for st2

18c1b61

Configure the system user and sudo access

8ae7c41

Configure system user in st2.conf

849f233

Add st2-web role

d3a6496

Add smoke tests

7968d1e

Make sure handlers are flushed before tests

85e7220

Updated playbook

9bdbd30

Fixed ssh_key bug in st2.conf

02ce665

Johan Dahlberg added 2 commits April 4, 2016 09:31

Purge instead of remove for Circle CI

c177e6e

It seems that the Circle CI still has old rabbit-mq configurations files lying around. Hopefully this will fix that.

configure mistral with sudo

03c058e

arm4b reviewed Apr 4, 2016
View reviewed changes

arm4b mentioned this pull request Apr 13, 2016

Ansible playbook needs an update #43

Closed

Removing meta and READMEs

1ef38da

arm4b reviewed May 2, 2016
View reviewed changes

Johan Dahlberg added 4 commits May 2, 2016 15:26

Get mistral stable by default

6cacd9c

rename st2-api -> st2api

855e019

Do not add st2 src repo

13598ee

Expire ssl cert after 365 days

f8ec21c

Remove warning from README

5994389

arm4b added the RFR label May 2, 2016

arm4b assigned enykeev May 2, 2016

arm4b added this to the v0.4.0 milestone May 2, 2016

enykeev reviewed May 5, 2016
View reviewed changes

Update restart/reload handlers

c182ee9

arm4b merged commit 36f63dc into StackStorm:master May 6, 2016

arm4b pushed a commit that referenced this pull request May 6, 2016

Docs cleanup for #42

2eb7854

Synchronize docs with recent #42 which installs new packages

arm4b mentioned this pull request May 6, 2016

Docs cleanup for #42 #44

Merged

arm4b pushed a commit that referenced this pull request May 6, 2016

Merge pull request #44 from StackStorm/feature/update-docs

34d8e92

Docs cleanup for #42

This was referenced May 6, 2016

Don't delegate version lookup to localhost #37

Closed

Add more Variables/Settings to Configure #20

Closed

arm4b unassigned enykeev May 6, 2016

johandahlberg deleted the use_deb_packages branch May 6, 2016 21:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating to use deb packages #42

Updating to use deb packages #42

johandahlberg commented Apr 1, 2016

manasdk commented Apr 1, 2016

johandahlberg commented Apr 1, 2016

manasdk commented Apr 1, 2016

johandahlberg commented Apr 4, 2016

arm4b commented Apr 4, 2016

arm4b Apr 4, 2016

enykeev commented Apr 4, 2016

johandahlberg commented Apr 4, 2016

enykeev commented Apr 5, 2016

arm4b May 2, 2016 •

edited

johandahlberg May 2, 2016

arm4b May 2, 2016

johandahlberg commented May 2, 2016

arm4b commented May 2, 2016

johandahlberg commented May 2, 2016

arm4b commented May 2, 2016

johandahlberg commented May 2, 2016

enykeev May 5, 2016

enykeev May 5, 2016

enykeev commented May 5, 2016

johandahlberg commented May 6, 2016

arm4b commented May 6, 2016

		@@ -1,4 +1,4 @@
		mistral_version: 0.13
		mistral_version: 1.3.2-99

Updating to use deb packages #42

Updating to use deb packages #42

Conversation

johandahlberg commented Apr 1, 2016

manasdk commented Apr 1, 2016

johandahlberg commented Apr 1, 2016

manasdk commented Apr 1, 2016

johandahlberg commented Apr 4, 2016

arm4b commented Apr 4, 2016

arm4b Apr 4, 2016

Choose a reason for hiding this comment

enykeev commented Apr 4, 2016

johandahlberg commented Apr 4, 2016

enykeev commented Apr 5, 2016

arm4b May 2, 2016 • edited

Choose a reason for hiding this comment

johandahlberg May 2, 2016

Choose a reason for hiding this comment

arm4b May 2, 2016

Choose a reason for hiding this comment

johandahlberg commented May 2, 2016

arm4b commented May 2, 2016

johandahlberg commented May 2, 2016

arm4b commented May 2, 2016

johandahlberg commented May 2, 2016

enykeev May 5, 2016

Choose a reason for hiding this comment

enykeev May 5, 2016

Choose a reason for hiding this comment

enykeev commented May 5, 2016

johandahlberg commented May 6, 2016

arm4b commented May 6, 2016

arm4b May 2, 2016 •

edited