-
-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update the Gitpod setup to use micromamba
to bootstrap the dev environment
#13030
Conversation
Thanks for making a pull request to jupyterlab! |
Not sure what's going on with the failing licenser check. |
Opened #13031 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some comments - @jtpio let me know if you have any questions or would like me to help with some of these changes/suggestions.
Also - while there is this work being done to add Gitpod (and it would reduce the friction for Jupyterlab dev and contributions), I would like to suggest adding docs to the developer docs such as https://numpy.org/devdocs/dev/development_gitpod.html
I can confirm that adding Gitpod + documentation has been a game-changer for contributions, sprints and even maintainers workflows for NumPy and SciPy, so worth a shot.
I am again happy to help with adding docs so ping me if you think this is a good addition.
@@ -1,40 +1,54 @@ | |||
github: | |||
prebuilds: | |||
# enable for the master/default branch (defaults to true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I generally prefer keeping these comments - though they might seem like visual noise they are helpful to provide context to folks who are not familiar with Gitpod config and might want to make improvements in the future
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I missed this one.
Sure let's keep them. Would you like to add them back in the follow-up documentation PR?
pushd /workspace | ||
wget -qO- https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba | ||
popd | ||
# bootstrap activation commands for other tasks to reuse | ||
cat <<EOT > /workspace/bin/activate-env.sh | ||
export MAMBA_ROOT_PREFIX=/workspace/.micromamba | ||
export MAMBA_EXE=/workspace/bin/micromamba | ||
$(/workspace/bin/micromamba shell hook --shell=bash) | ||
export JUPYTER_PREFER_ENV_PATH=1 | ||
micromamba activate | ||
EOT | ||
source /workspace/bin/activate-env.sh | ||
micromamba config append channels conda-forge | ||
micromamba install -n base -y python=3.10 nodejs=14 yarn | ||
python -m pip install -e ".[dev,docs,test]" && jlpm install && jlpm run build | ||
gp sync-done setup |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since all these extras to use mamba are being added, I would highly recommend specifying a custom Dockerfile (see https://www.gitpod.io/docs/config-docker#configure-a-custom-dockerfile) this would help keep this config file cleaner and also removes the need to add later the echo "source /workspace/bin/activate-env.sh" >> ~/.bashrc
command in line 40.
Plus, we can ensure that the corresponding environment is always activated by default and reduce the need to have all the gp sync
related commands.
This is the approach currently used in NumPy and SciPy. This setup and leveraging prebuilds ensures that the Gitpod instance is pretty much ready to use once someone starts a new workspace.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, I think the main motivation for proceeding like this was to keep things as simple as possible and self-contained in one file. It's not clear whether folks are actively using Gitpod at the moment, this didn't seem to be the case when the question was raised at a JupyterLab weekly call a few weeks ago.
Switching to a Dockerfile
sounds good, although Docker can also be complicated and requires extra knowledge to maintain. Although it could also pave the way to support GitHub Codespaces at some point to with devcontainer
.
Something else - I'd also suggest adding a couple of extensions (i.e. Python language support, prettier) - as it also helps with developer experience (see https://github.com/numpy/numpy/blob/main/.gitpod.yml for a number of extensions added during the prebuild) |
Many thanks @trallard for the review and the help!
I personally think it would be great to have. The main question raised before was about maintaining this documentation if it's not actively used (might become outdated after a while). But if we are at least two then can probably add that yes 👍 |
Ok will get started with a PR then After seeing interns and folks onboarding to JLab and being tripped while setting up their dev environment i am convinced this will be very handy. And I am happy to help with maintenance shall it be needed |
Thanks @trallard. Since there is a sprint day tomorrow at EuroScipy and some folks might be interested in contributing to JupyterLab, maybe we can merge this PR and address the comments above in follow-up PRs? (or track them in an issue) |
Sure thing - sorry I missed this notification until now |
OK thanks, then getting this in so we can iterate on it. |
Just checked the config on the This seems to require building and pushing a custom Docker image in https://github.com/numpy/numpy/blob/main/.github/workflows/gitpod.yml. Probably this could also be made compatible with GitHub Codespaces then. |
References
Follow-up to #12697
The goal is to help onboard new contributors and get passed the "setup a dev environment locally" step as quickly as possible so they can go to the interesting part faster.
This is also motivated by in-person sprints where folks want to be productive quickly, and might actually never contribute again in the future (although we would like for them to stick around!). So doing all the dev setup locally might not be worth it.
Code changes
Update
.gitpod.yml
to:micromamba
to create the environment and install the base dependencies.gitpod.yml
so it's self containedWhen a Gitpod workspace is stopped and started again, only the content under
/workspaces
is kept. Which is why we save themicromamba
environments under/workspaces/.micromamba
so all the depedencies are available on startup.User-facing changes
Backwards-incompatible changes