Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate Docker containers to off-docker QEMU machine #62

Closed
43 of 44 tasks
ocervell opened this issue Oct 7, 2021 · 1 comment
Closed
43 of 44 tasks

Migrate Docker containers to off-docker QEMU machine #62

ocervell opened this issue Oct 7, 2021 · 1 comment
Assignees
@ocervell
Labels
🐳 docker ✨ enhancement New feature or request

Comments

@ocervell
Copy link
Contributor

ocervell commented Oct 7, 2021 •
edited by alexgarel
Loading

Goal: Migrate Docker containers running on Proxmox VMs to new QEMU machine after .net outage

Tasks:

  • QEMU VM Scripts / Commands (manual):

    • Install Docker
    • Switch Docker storage driver to overlay2, recommended for production
    • Create off user on the VM
    • Mount ZFS clones in /mnt/products, /mnt_users, /mnt/orgs and add to /etc/fstab
    • Correct chown permissions for /mnt/products, /mnt/users, /mnt/orgs to off:off
    • Add data dumps to /mnt/data
    • Rsync /mnt/product_images

  • Docker Compose deployments (manual)

    • mongo-dev
    • robotoff-net
    • robotoff-ann
    • robotoff-ml
    • off-net
    • monitoring

  • MongoDB daily tasks (manual):

    • Import production data into MongoDB (manual)
    • Refresh product tags (manual)

  • Switch GitHub environments secrets to new host for deployments:

    • openfoodfacts-server: HOST=10.1.0.200, USER=off, SSH_PRIVATE_KEY=<off_private_key>, MONGODB_HOST=10.1.0.200
    • robotoff: HOST=10.1.0.200, USER=off, SSH_PRIVATE_KEY=<off_private_key>, POSTGRES_HOST=10.1.0.200, MONGO_URI=mongodb://10.1.0.200:27017, INFLUXDB_HOST=10.1.0.200, ELASTICSEARCH_HOSTS=10.1.0.200
    • robotoff-ann: HOST=10.1.0.200, USER=off, SSH_PRIVATE_KEY=<off_private_key>
    • impactestimator: HOST=10.1.0.200, USER=off, SSH_PRIVATE_KEY=<off_private_key>
    • monitoring: HOST=10.1.0.200, USER=off, SSH_PRIVATE_KEY=<off_private_key>

  • Point NGINX routes to new VM Docker containers:

    • openfoodfacts.net --> 10.1.0.200:80 # OFF
    • robotoff.openfoodfacts.net/api/v1/ann/* --> 10.1.0.200:5501 # Robotoff ANN
    • robotoff.openfoodfacts.net/api/v1/* --> 10.1.0.200:5500 # Robotoff
    • monitoring.openfoodfacts.net/alerts* --> 10.1.0.200:9115 # AlertManager
    • monitoring.openfoodfacts.net/metrics* --> 10.1.0.200:9090 # Prometheus
    • monitoring.openfoodfacts.net --> 10.1.0.200:3000 # Grafana

  • PRs for automated deployments updates:

  • Run automated deployments successfully:

    • robotoff-net
    • robotoff-ml
    • robotoff-ann
    • mongo-dev
    • off-net
    • monitoring
    • impactestimator-net
@ocervell ocervell self-assigned this Oct 7, 2021
@ocervell ocervell added the ✨ enhancement New feature or request label Oct 7, 2021
@alexgarel
Copy link
Member

This one is all done since a long time ! Thanks again @ocervell :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ocervell ocervell

Labels
🐳 docker ✨ enhancement New feature or request
Projects
Infrastructure & Monitoring
Archived in project
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexgarel @teolemon @ocervell