Add allen-swdb hub #1585
Add allen-swdb hub #1585
Conversation
Support and Staging deployments
Production deployments
|
| - enc-support.secret.values.yaml | ||
| hubs: | ||
| - name: staging | ||
| display_name: "Staging" |
There was a problem hiding this comment.
| display_name: "Staging" | |
| display_name: "Allen Institute (staging)" |
| ```bash | ||
| terraform output -raw continuous_deployer_creds > ../../config/clusters/<your-cluster-name>/deployer-credentials.secret.json | ||
| sops --output config/clusters/<your-cluster-name>/enc-deployer-credentials.secret.json --encrypt ../../config/clusters/<your-cluster-name>/deployer-credentials.secret.json | ||
| terraform output -raw continuous_deployer_creds > ../../config/clusters/<your-cluster-name>/enc-deployer-credentials.secret.json | ||
| sops --in-place --encrypt ../../config/clusters/<your-cluster-name>/enc-deployer-credentials.secret.json | ||
| ``` |
There was a problem hiding this comment.
Maybe we can keep the original command and provide your suggestion as an alternative? I like to not use --in-place and explicitly add the enc- through --output because our .gitignore file prevents files that contain secret in their name but are not prefixed with enc- from being accidentally committed to the repo. I can then clean up unencrypted files using git clean.
https://infrastructure.2i2c.org/en/latest/topic/secrets.html
There was a problem hiding this comment.
@sgibson91 sorry Sarah, I know this isn't the first time I'd changed this to --in-place - I had forgotten the last time you had reminded me of it! I've just reverted this change to go back to using --output instead as it was before, and will keep this in mind for the future!
| GitHubOAuthenticator: | ||
| allowed_organizations: | ||
| - 2i2c-org:tech-team | ||
| - alleninstitute |
There was a problem hiding this comment.
Please add authorization for GitHub Org SWDB2022: https://github.com/SWDB2022
| initContainers: | ||
| # Need to explicitly fix ownership here, since EFS doesn't do anonuid | ||
| - name: volume-mount-ownership-fix | ||
| image: busybox | ||
| command: | ||
| [ | ||
| "sh", | ||
| "-c", | ||
| "id && chown 1000:1000 /home/jovyan /home/jovyan/shared && ls -lhd /home/jovyan", | ||
| ] | ||
| securityContext: | ||
| runAsUser: 0 | ||
| volumeMounts: | ||
| - name: home | ||
| mountPath: /home/jovyan | ||
| subPath: "{username}" | ||
| - name: home | ||
| mountPath: /home/jovyan/shared | ||
| subPath: _shared |
There was a problem hiding this comment.
This part can go after #1591 gets merged, right?
There was a problem hiding this comment.
@GeorgianaElena yep! I just merged that, will rebase and get rid of this!
| # Expllicitly unset mem_limit, so it overrides the default memory limit we set in | ||
| # basehub/values.yaml | ||
| mem_limit: null |
There was a problem hiding this comment.
So what this means is that there a mem guarantee, but no upper limit, right? Can I ask what's the reasoning behind this? (I'm planning to open an issue about resource allocation best practices and I'm gathering info 😅 )
|
I'm gonna work on getting this merged! Things to do:
|
yuvipanda
force-pushed
the
allen-swdb
branch
2 times, most recently
from
d9718f4
to
d922e97
Compare
There was a problem hiding this comment.
I have a small suggestion regarding CLI arguments. Also, the deployer's README should be updated
deployer/cli.py
Outdated
| generate_cluster_parser.add_argument( | ||
| "--cloud-provider", | ||
| choices=["aws"], | ||
| help="Which cloud provider to generate a cluster for", | ||
| ) |
There was a problem hiding this comment.
I would make this a required argument, rather than optional, because we will always have to specify it and typing out --cloud-provider would get repetitive
|
|
||
| ```bash | ||
| sops --in-place --encrypt eksctl/ssh-keys/<cluster-name>.key | ||
| python3 deployer generate-cluster --cloud-provider aws <cluster-name> |
There was a problem hiding this comment.
See above suggestion to make --cloud-provider a required arg
|
@yuvipanda, do you think we can get this one merged soon (once the review comments and the conflict are fixed)? |
|
@damianavila yes i'll try to get that done asap. |
|
Is this PR ready to be merged? I believe this is the last step to close this issue. |
There are still some comments to be addressed and branch conflicts to be resolved before merging it. |
|
@yuvipanda confirmed to me he is going to take a look at the provided feedback and merge this one soon (most likely early next week). |
- Also add a generate-cluster command to our deployer that generates the jsonnet & tfvars files from a template Ref 2i2c-org#1440
Turns out there is literally one thing that's valid JSON but not YAML - hard tabs. Guess what golang's JSON outputter loves?!
Co-authored-by: Sarah Gibson <44771837+sgibson91@users.noreply.github.com>
We use jupyterhub-roothooks in the container to drop privileges after starting as root and running s3fs mounting code.
Per request!
Matches what we pass in allowed_organizations in GitHubOAuthenticator
yuvipanda
force-pushed
the
allen-swdb
branch
from
d922e97
to
fe1c9ea
Compare
Co-authored-by: Sarah Gibson <44771837+sgibson91@users.noreply.github.com>
|
I've made all the changes suggested by @sgibson91, so gonna merge this one (and immediately decom it). Apologies everyone that this took so long! |
|
🎉🎉🎉🎉 Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/3161567755 |
the jsonnet & tfvars files from a template
Ref #1440