Add support for Ubuntu Xenial in development #3108

hectcastro · 2019-06-12T23:13:03Z

Overview

Demo

Notes

Analysis jobs completed, but almost always returned table cells with 0s
Delineate watershed did not work because I do not have the necessary data volume
Tile service is not functional
CI hangs on one of the Testem tests.

Testing Instructions

Destroy all of the local virtual machines and clear out the azavea.* roles
Follow the instructions in the README to setup the local development environment

$ vagrant up
$ ./scripts/manage.sh migrate

From within the app virtual machine:

$ cd /vagrant
./scripts/setupdb.sh -b  # These can take a few minutes
./scripts/setupdb.sh -s

Interact with the application to determine what functionality is not working properly

- Migrate service to systemd - Centralize logs in syslog via systemd - Update dependencies (Things are broken due to old version of Windshaft.)

hectcastro · 2019-06-20T12:56:44Z

README.md

 - Redis
- Logstash


All of the log and metrics aggregation services have been removed. Follow-up tasks exist to replace these with external services.

hectcastro · 2019-06-20T12:57:09Z

README.md

@@ -12,12 +12,7 @@ A combination of Vagrant 1.6+ and Ansible 1.8+ is used to setup the development
 The `app` virtual machine contains an instance of the Django application, `services` contains:

 - PostgreSQL
- Pgweb


pgweb was not consistently being used, so it was removed from the development environment.

hectcastro · 2019-06-20T12:57:21Z

README.md

@@ -28,7 +23,6 @@ The `app` virtual machine contains an instance of the Django application, `servi

 - Celery
 - Docker
- Spark Job Server (container)


This service has not been around for a while, and was not removed in this PR.

hectcastro · 2019-06-20T12:57:53Z

README.md


 ### Caching

-In order to speed up things up, you may want to consider leveraging the `vagrant-cachier` plugin. If installed, it is automatically used by Vagrant.


This plugin is no longer actively maintained and is not installed is most environments where this Vagrant setup is instantiated.

hectcastro · 2019-06-20T12:58:57Z

Vagrantfile

@@ -89,7 +68,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    worker.vm.hostname = "worker"
    worker.vm.network "private_network", ip: ENV.fetch("MMW_WORKER_IP", "33.33.34.20")

-    worker.vm.synced_folder "src/mmw", "/opt/app/"
+    worker.vm.synced_folder "src/mmw", "/opt/app"


This was done to trim the trailing _ from mount names in df output. Mount names also coincide with systemd targets.

hectcastro · 2019-06-20T16:04:55Z

deployment/ansible/roles/model-my-watershed.tiler/tasks/dependencies.yml

-    - "libjpeg8-dev=8c*"
-    - "libgif-dev=4.1.*"
+  apt:
+    pkg: ["libcairo2-dev=1.14.*",


Ubuntu 16.04 specific package versions.

hectcastro · 2019-06-20T16:06:51Z

deployment/ansible/roles/model-my-watershed.tiler/templates/nginx-tiler.conf.j2

@@ -2,7 +2,8 @@ server {
  listen *:80;
  server_name _;

-  access_log /var/log/nginx/mmw-tiler.access.log logstash_json;
+  error_log syslog:server=unix:/dev/log,facility=local7,tag=nginx_tiler,severity=error;


Write directly to syslog from Nginx.

hectcastro · 2019-06-20T16:07:49Z

deployment/cfn/application.py

@@ -17,14 +17,11 @@
 from utils.constants import (
    ALLOW_ALL_CIDR,
    EC2_INSTANCE_TYPES,
-    GRAPHITE,


All of the changes in this file should really not be in this PR because they are staging specific concerns. I ended up removing them because I was interactively going through find/replace.

Actually testing that these changes are working should be deferred to another PR creation/review cycle.

hectcastro · 2019-06-20T16:09:06Z

src/mmw/apps/core/mail/backends/boto_ses_mailer.py

@@ -13,7 +13,6 @@
 from django.core.mail.backends.base import BaseEmailBackend
 from django.core.exceptions import ImproperlyConfigured

-from django_statsd.clients import statsd


Statsite/StatsD has been removed, so application level libraries interacting with those services has been removed as well.

src/mmw/apps/core/mail/backends/boto_ses_mailer.py

hectcastro · 2019-06-21T03:05:22Z

Provisioning in a fresh check-out failed for the worker with the following error:

RUNNING HANDLER [azavea.nginx : Restart Nginx] *********************************
Thursday 20 June 2019  16:55:13 -0400 (0:00:00.104)       0:09:35.197 *********
fatal: [worker]: FAILED! => {"changed": false, "msg": "Unable to start service nginx: Job for nginx.service failed because the control process exited with error code. See \"systemctl status nginx.service\" and \"journalctl -xe\" for details.\n"}

I believe this was fixed in 12928c0.

rajadain

+1 tested, everything's working (tested Analysis, Monitoring, Modeling, which exercised Django, Ulmo, Celery, and the Geoprocessing Service) except for the tile service, but that's expected. I think we should detach that from this PR and let #3109 handle it.

Thanks for all the explanatory comments in the PR, and also for updating the README to match the new state.

Regarding the CI failure, I don't think it's a specific front-end test, but that the front-end testing doesn't exit when all the tests are run. Feel free to defer that to a later card. With this work in we can begin all the other upgrades.

Excellent work 👏 🙇

hectcastro · 2019-06-21T21:12:14Z

TY. I agree with deferring Windshaft things to #3109. I also opened #3110 to deal with the Testem test suite.

I'm going to take a closer look at the service logging aspects of this PR in another pass, then merge. Thanks for the end-to-end testing.

To get the dev environment in a working state. This also removes most monitoring tooling, since that is not needed in development. Much of this work was inspired by WikiWatershed/model-my-watershed#3108

hectcastro added 14 commits June 12, 2019 13:55

Attempt to remove all monitoring specific configuration

fa47ec7

Get services VM to provision cleanly

96f1ef0

Get tiler VM provisioning cleanly

4dfe5b7

- Migrate service to systemd - Centralize logs in syslog via systemd - Update dependencies (Things are broken due to old version of Windshaft.)

App provisions cleanly and services work?

c0f36b4

Delete default Nginx site across other services

d3a624f

Template file naming consistency

370fbc4

Fix Ansible deprecation warnings

c8f44f7

Bump version of Docker to 18.x

54e7b96

Consistency

a64fd70

Convert celeryd to a systemd service

d8011d1

Convert geoprocessing to a systemd service

7a02193

Convert RWD to a Docker service

379d4bc

Fix worker Vagrantfile

fc0917c

Update README.md

13aafee

hectcastro commented Jun 20, 2019

View reviewed changes

hectcastro marked this pull request as ready for review June 20, 2019 16:23

hectcastro requested a review from rajadain June 20, 2019 16:23

This comment has been minimized.

Sign in to view

hectcastro added 2 commits June 20, 2019 23:03

Add syslog logging to geop Nginx configuration

12928c0

Remove stale comment

4505d49

rajadain approved these changes Jun 21, 2019

View reviewed changes

rajadain assigned hectcastro Jun 21, 2019

hectcastro mentioned this pull request Jun 21, 2019

Address issues where Testem suite does not exit after all tests are run #3110

Closed

hectcastro added 2 commits June 21, 2019 18:10

Fix flawed WantedBy mount names

d9323be

Remove _log_quota method

b6606d6

hectcastro changed the title ~~Add support for Ubuntu Xenial (16.04)~~ Add support for Ubuntu Xenial in development Jun 21, 2019

hectcastro merged commit e70fef2 into develop Jun 21, 2019

hectcastro deleted the feature/hmc/ubuntu-16.04 branch June 24, 2019 12:58

rajadain added the WPF Funding Source: William Penn Foundation label Jun 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Ubuntu Xenial in development #3108

Add support for Ubuntu Xenial in development #3108

hectcastro commented Jun 12, 2019 •

edited

Loading

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

hectcastro Jun 20, 2019

This comment has been minimized.

This comment has been minimized.

hectcastro commented Jun 21, 2019

rajadain left a comment

hectcastro commented Jun 21, 2019


		### Caching

		In order to speed up things up, you may want to consider leveraging the `vagrant-cachier` plugin. If installed, it is automatically used by Vagrant.

Add support for Ubuntu Xenial in development #3108

Add support for Ubuntu Xenial in development #3108

Conversation

hectcastro commented Jun 12, 2019 • edited Loading

Overview

Demo

Notes

Testing Instructions

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

hectcastro commented Jun 21, 2019

rajadain left a comment

Choose a reason for hiding this comment

hectcastro commented Jun 21, 2019

hectcastro commented Jun 12, 2019 •

edited

Loading