Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Maintenance] Migrate Docker Cluster stack over to Dallas-Makerspace/datagrid stack #543

Open
denzuko opened this issue Aug 21, 2019 · 6 comments

Comments

@denzuko
Copy link
Member

@denzuko denzuko commented Aug 21, 2019

Edge CDN

  • put communitygrid.dallasmakerspace.org into maintenance mode
  • Pause uptime.dallasmakerspace.org
  • Take communitygrid.dallasmakerspace.org out of maintenance mode
  • Resume uptime.dallasmakerspace.org

Networking

  • Enable dhcp on subnet
  • Pull nodes out of load balancer
  • Configure Sticky sessions
  • Join managers in load balancer by dns entry
  • configure failover test for http://managerIp/api/status to return => HTTP/2 200

Workers

  • Pull workers from cluster and upgrade to latest docker version
  • prlxcompute01-6 moved from virtwin1 to prvir02
  • Apply Host OS hotfixes and security patches
  • Join workers back in cluster
  • Deploy zerotier-one
  • Join to management network
  • Configure ssh allow from management network
  • Install ssh key only authentication
  • Configure pam_ssh_agent_auth for sudo

Managers

  • Pull motherboard from cluster
  • Apply Host OS hotfixes and security patches
  • Join motherboard with cluster
  • Pull killbill from cluster
  • Apply Host OS hotfixes and security patches
  • Join killbill with cluster
  • Pull dallas-makerspace/datagrid
  • Apply secrets to config
  • Deploy stack
  • Deploy 'docker system prune -af' as a dind service named sysprune with global deploy
  • Apply portainer customizations
  • Add docker update -f sysprune
  • Deploy zerotier-one
  • Join to management network
  • Configure ssh allow from management network
  • Install ssh key only authentication
  • Configure pam_ssh_agent_auth for sudo
  • Prune old services that have not been updated past 90 days
  • Run actuary security tests
  • Run rootkit hunter
  • Enable fail2ban on ssh

Github

  • Update endpoint from communitygrid.dallasmakerspace.org to admin.dallasmakerspace.org

Validation

  • should display chronograf on monitoring.dallasmakerspace.org
  • should display portainer on communitygrid.dallasmakerspace.org
  • should display home assistant on dashboard.dallasmakerspace.org
  • should display traefik dashboard on loadbalancer.dallasmakerspace.org
  • should display jupyter notebooks dashboard on notebook.dallasmakerspace.org
  • should display fbctf on ctf.dallasmakerspace.org
  • should display vcc wiki page on vcc.dallasmakerspace.org
  • should display radio stream on radio.dallasmakerspace.org
  • should display fancy status pages on testshouldfailwith404.dallasmakerspace.org and communitygrid.dallasmakerspace.org
  • should have functional github authentication on admin.dallasmakerspace.org
  • all managers should be accessible from firewall
  • all nodes should be accessible from zerotier
  • MQTT broker should be accessible from broker.dallasmakerspace.org
  • Should display gitea on git.dallasmakerspace.org
  • Should display drone on drone.dallasmakerspace.org
  • Should display hashicorp vault on vault.dallasmakerspace.org
  • uptime.dallasmakerspace.org should be green
@gitcoinbot

This comment has been minimized.

Copy link

@gitcoinbot gitcoinbot commented Aug 21, 2019

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


This issue now has a funding of 0.108 ETH (20.07 USD @ $185.8/ETH) attached to it as part of the Dallas Makerspace fund.

@denzuko

This comment has been minimized.

Copy link
Member Author

@denzuko denzuko commented Aug 24, 2019

Had to roll back changes to acquire access to admin wiki. (again)

@janus

This comment has been minimized.

Copy link

@janus janus commented Sep 14, 2019

This looks interesting ... but I wonder where to get started ?

@denzuko

This comment has been minimized.

Copy link
Member Author

@denzuko denzuko commented Oct 11, 2019

@janus one would need to be on the @Dallas-Makerspace/infrastructure team to implement the changes or even access the servers.

@janus

This comment has been minimized.

Copy link

@janus janus commented Oct 11, 2019

@denzuko Thanks . I am not part of the team

@stale

This comment has been minimized.

Copy link

@stale stale bot commented Nov 10, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Nov 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.