Clean up CONP VM environment

[shots47s]

Purpose

There is a need to reorganize and clean up our current set of CONP VMs. Currently each have been deployed with a single use account that is being shared amongst several people, and each of them doesn't at this point serve a clear purpose.

In order for the project to move from a demo platform to a full fledged production environment with a geographically shared development effort, we need to bring these into the MCIN fold and reevaluate how our developers interact with the infrastructure.

Context

As we are moving into taking the CONP development from a demonstration effort to a long-term sustained production environment, we need to reevaluate how we securely and appropriately utilize MCIN infrastructure. Ideally, the VMs created should be for testing and hosting deployments rather than as shared environments for development (as this is not a scalable solution, especially for remote development). Additionally, it is incredibly insecure to have shared VMs with only one user account that are not tied to centralized user accounting system, as auditing activity is impossible.

Possible Implementation (optional) (not sequential)

1. Audit current set of VM's deployed for CONP and determine their purpose and need.
2. Tie all existing VMs to the MCIN LDAP with appropriate groupings to facilitate who needs what access to the VMs (e.g. who needs sudo).
3. Ensure all developers that need access to MCIN VMs are in the LDAP and placed in appropriate groups.
4. Develop a plan for on-boarding new developers with in mind that many may be geographically distributed and will need to have local environments set up.

Related issues (optional)

#69

[andytengca]

I am going to send the request to change DNS record.

1. change dev VMs to portal-dev.conp.ca
2. create DNS for prod VMs portal.conp.ca

will let you know once done.

[glatard]

Sorry to chime in late on that, but I'm not sure if we should provide a development infrastructure to developers. Instead, it would be easier if developers could have their own installation of the portal and develop there. This has worked for @JoeyZhou when developing the pipeline part of the code, up to minor configuration issues.

[shots47s]

I agree 100%, these VMs were more setup for Datalad stuff, although we should move to make that as distributed as possible as well, otherwise, we really aren't using Datalad correctly. These are being setup now mostly to provide deployment servers.

[shots47s]

@andytengca, can you put a list of the VMs available to the project here so that we have them and then I can close this issues?

[andytengca]

We have 3 dev VMs as below and I also changed the DNS record. You can access the below 2 VMs via your ldap credentials ssh -p 30032 <username>@portal-dev.conp.ca ssh -p 33031 <username>@accounts-dev.conp.ca also DB for portal-dev.conp.ca is on mariadb-dev.acelab.ca (MariaDB 10.3 dev cluster) please refer to redmine ticket : https://redmine.cbrain.mcgill.ca/issues/16852 And I don't know if you still need datalad-dev VM? if not, please confirm we can shutdown the VM. Thanks, Andy On 2019-08-29 1:37 p.m., Shawn Brown wrote: @andytengca<https://github.com/andytengca>, can you put a list of the VMs available to the project here so that we have them and then I can close this issues? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#70?email_source=notifications&email_token=AH4SS3KPYGCBGJ5OWNZZ3S3QHACNPA5CNFSM4ILDYNLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5PIERY#issuecomment-526287431>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AH4SS3LN7FMM4QUCO74H7NLQHACNPANCNFSM4ILDYNLA>.

[shots47s]

Thanks @andytengca, this is super helpful. I am not sure what we would use datalad.conp.ca, so for now, you can decomission. Can we start it back up if we have a need for it?

[andytengca]

Sure. I will decommission the datalad-dev.conp.ca all the portal-dev.conp.ca and accouts-dev.conp.ca VMs are backing up on a weekly basis and we keep recent 2 backups. DB on portal-dev.conp.ca (called conp_portal_dev) is backing up on a daily basic and we keep the recent 6 backups. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#70?email_source=notifications&email_token=AH4SS3PPXZHIUDD7IE4VEMLQHEUHDA5CNFSM4ILDYNLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5RZNUY#issuecomment-526620371>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AH4SS3IZZ4XICWW4KL4GHZTQHEUHDANCNFSM4ILDYNLA>.