Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move edxapp vars to app #89

Merged
merged 7 commits into from Jul 27, 2018
Merged

Move edxapp vars to app #89

merged 7 commits into from Jul 27, 2018

Conversation

jmaupetit
Copy link
Contributor

@jmaupetit jmaupetit commented Jul 25, 2018

Purpose

This is the second step of our application driven migration, now targeting the edxapp application

Proposal

  • fix app/services labels in templates
  • define an host+port for the patient0/development project
  • move edxapp variables from group_vars/all/main.yml to apps/edxapp/vars/all/main.yml and use them in templates
  • all synchronous sequential job execution 🎉
  • test edxapp's deployment in the CI
  • wait for all services to be up before running jobs
  • add support for multiple hosts per-service edit: postponed to another PR (see Add support for multiple host per-service #90)
  • add lms host definition and test it edit: postponed to another PR (see Add support for multiple host per-service #90)

@jmaupetit jmaupetit self-assigned this Jul 25, 2018
@jmaupetit jmaupetit force-pushed the move-edxapp-vars-to-app branch 3 times, most recently from 323005b to 08d241c Compare July 27, 2018 13:46
@@ -2,17 +2,19 @@ apiVersion: v1
kind: Service
metadata:
labels:
app: edxapp-lms
version: "{{ edxapp_tag }}"
app: lms
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be edxapp?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, well spotted!

bin/dev Outdated
@@ -2,88 +2,42 @@
#
# Arnold's dev script
#
# We use KVM as the default hypervisor, if you want to use VirtualBox instead,
# please set the USE_KVM environment variable to 0, e.g.:
# Berore running this script, please make sure that your system met the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Berore/Before

bin/dev Outdated
# "insecure-registries" : ["172.30.0.0/16"]
# }
#
# 2. To run ES, you will to ensure that your kernel's vm.max_map_count
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you will NEED to ensure

function _login() {
_set_minishift_path
oc login "https://$(minishift ip):8443" --username=developer --password=developer
oc cluster up --server-loglevel=5 --public-hostname="${OPENSHIFT_DOMAIN}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a bit more than login!?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is no longer called login. Forget about this crappy diff.

cms_port: 8081
lms_port: 8071
# cms_port: 8081
# lms_port: 8071
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dead code

# mongodb_image: "centos/mongodb-32-centos7"
# mongodb_tag: "3.2"
# mysql_image: "centos/mysql-57-centos7"
# mysql_tag: "5.7"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dead code

# port: "27017"
# mysql:
# host: "mysql"
# port: "3306"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dead code

Move edxapp specific variables to the edxapp defaults and use them in
edxapp templates. edxapp templates have been cleanup for cross
applications consistency (labels, etc.).
We are now able to run jobs in a particular order depending on their
template file name: jobs are sorted by their filename for an application
(all services), e.g.:

1. apps/edxapp/templates/cms/job_01_collectstatic.yml.j2
2. apps/edxapp/templates/lms/job_02_collectstatic.yml.j2
3. ...

And jobs are now executed sequentially, e.g. a new job cannot be
submitted if the previous one in the list did not complete.
The CI VM needs to be able to run a dockerized ES instance so that
Richie's "regenerate index" job can run properly.
edxapp(cms) deployment is now also tested in the CI \o/
To prevent jobs failure due to service unavailability, we've added a
wait loop that ensures that all services have been created first.
The bin/dev script now uses the "oc cluster" command to start a local
openshift cluster instead of minishift. This lowers down the
requirements and configuration time to start working with Arnold (and it
drastically improves the cluster performances).

The minishift start script has been kept and moved to bin/minishift for
those who would still prefer this legacy solution.

Fix #81
Running edxapp migrations is way too long (more than 20 minutes). Hence
we decided to speed up edxapp deployment by adding a new job that loads
a sql dump of the database schema before running migrations.
@jmaupetit jmaupetit changed the title WIP: Move edxapp vars to app Move edxapp vars to app Jul 27, 2018
@jmaupetit jmaupetit removed the WIP label Jul 27, 2018
@jmaupetit
Copy link
Contributor Author

We still have a timeout from CircleCI's test-bootstrap job. We decided to merge this as-is and address this timeout issue in a new PR: the idea is to split the test-bootstrap job in a per-app job, to hopefully prevent job failure due to a timeout issue.

@jmaupetit jmaupetit merged commit b4ef7fe into master Jul 27, 2018
@jmaupetit jmaupetit deleted the move-edxapp-vars-to-app branch July 27, 2018 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants