Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the Gitpod setup to use micromamba to bootstrap the dev environment #13030

Merged
merged 4 commits into from
Sep 2, 2022

Conversation

jtpio
Copy link
Member

@jtpio jtpio commented Aug 30, 2022

References

Follow-up to #12697

The goal is to help onboard new contributors and get passed the "setup a dev environment locally" step as quickly as possible so they can go to the interesting part faster.

This is also motivated by in-person sprints where folks want to be productive quickly, and might actually never contribute again in the future (although we would like for them to stick around!). So doing all the dev setup locally might not be worth it.

Code changes

Update .gitpod.yml to:

  • Use micromamba to create the environment and install the base dependencies
  • Auto activate the environment when opening new terminals
  • Keep everything in .gitpod.yml so it's self contained

When a Gitpod workspace is stopped and started again, only the content under /workspaces is kept. Which is why we save the micromamba environments under /workspaces/.micromamba so all the depedencies are available on startup.

User-facing changes

Backwards-incompatible changes

@jtpio jtpio added this to the 4.0.0 milestone Aug 30, 2022
@jupyterlab-probot
Copy link

Thanks for making a pull request to jupyterlab!
To try out this branch on binder, follow this link: Binder

@jtpio
Copy link
Member Author

jtpio commented Aug 30, 2022

Looking good:

image

@jtpio jtpio marked this pull request as ready for review August 30, 2022 16:35
@jtpio
Copy link
Member Author

jtpio commented Aug 30, 2022

Not sure what's going on with the failing licenser check.

@jtpio
Copy link
Member Author

jtpio commented Aug 30, 2022

Not sure what's going on with the failing licenser check.

Opened #13031

Copy link
Contributor

@trallard trallard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments - @jtpio let me know if you have any questions or would like me to help with some of these changes/suggestions.

Also - while there is this work being done to add Gitpod (and it would reduce the friction for Jupyterlab dev and contributions), I would like to suggest adding docs to the developer docs such as https://numpy.org/devdocs/dev/development_gitpod.html
I can confirm that adding Gitpod + documentation has been a game-changer for contributions, sprints and even maintainers workflows for NumPy and SciPy, so worth a shot.

I am again happy to help with adding docs so ping me if you think this is a good addition.

@@ -1,40 +1,54 @@
github:
prebuilds:
# enable for the master/default branch (defaults to true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally prefer keeping these comments - though they might seem like visual noise they are helpful to provide context to folks who are not familiar with Gitpod config and might want to make improvements in the future

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I missed this one.

Sure let's keep them. Would you like to add them back in the follow-up documentation PR?

Comment on lines +14 to 29
pushd /workspace
wget -qO- https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
popd
# bootstrap activation commands for other tasks to reuse
cat <<EOT > /workspace/bin/activate-env.sh
export MAMBA_ROOT_PREFIX=/workspace/.micromamba
export MAMBA_EXE=/workspace/bin/micromamba
$(/workspace/bin/micromamba shell hook --shell=bash)
export JUPYTER_PREFER_ENV_PATH=1
micromamba activate
EOT
source /workspace/bin/activate-env.sh
micromamba config append channels conda-forge
micromamba install -n base -y python=3.10 nodejs=14 yarn
python -m pip install -e ".[dev,docs,test]" && jlpm install && jlpm run build
gp sync-done setup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since all these extras to use mamba are being added, I would highly recommend specifying a custom Dockerfile (see https://www.gitpod.io/docs/config-docker#configure-a-custom-dockerfile) this would help keep this config file cleaner and also removes the need to add later the echo "source /workspace/bin/activate-env.sh" >> ~/.bashrc command in line 40.
Plus, we can ensure that the corresponding environment is always activated by default and reduce the need to have all the gp sync related commands.
This is the approach currently used in NumPy and SciPy. This setup and leveraging prebuilds ensures that the Gitpod instance is pretty much ready to use once someone starts a new workspace.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I think the main motivation for proceeding like this was to keep things as simple as possible and self-contained in one file. It's not clear whether folks are actively using Gitpod at the moment, this didn't seem to be the case when the question was raised at a JupyterLab weekly call a few weeks ago.

Switching to a Dockerfile sounds good, although Docker can also be complicated and requires extra knowledge to maintain. Although it could also pave the way to support GitHub Codespaces at some point to with devcontainer.

@trallard
Copy link
Contributor

Something else - I'd also suggest adding a couple of extensions (i.e. Python language support, prettier) - as it also helps with developer experience (see https://github.com/numpy/numpy/blob/main/.gitpod.yml for a number of extensions added during the prebuild)

@jtpio
Copy link
Member Author

jtpio commented Aug 31, 2022

Many thanks @trallard for the review and the help!

I am again happy to help with adding docs so ping me if you think this is a good addition.

I personally think it would be great to have. The main question raised before was about maintaining this documentation if it's not actively used (might become outdated after a while). But if we are at least two then can probably add that yes 👍

@trallard
Copy link
Contributor

Ok will get started with a PR then

After seeing interns and folks onboarding to JLab and being tripped while setting up their dev environment i am convinced this will be very handy. And I am happy to help with maintenance shall it be needed

@jtpio
Copy link
Member Author

jtpio commented Sep 1, 2022

Thanks @trallard.

Since there is a sprint day tomorrow at EuroScipy and some folks might be interested in contributing to JupyterLab, maybe we can merge this PR and address the comments above in follow-up PRs? (or track them in an issue)

@trallard
Copy link
Contributor

trallard commented Sep 1, 2022

Sure thing - sorry I missed this notification until now

@jtpio
Copy link
Member Author

jtpio commented Sep 2, 2022

OK thanks, then getting this in so we can iterate on it.

@jtpio jtpio merged commit af70f3a into jupyterlab:master Sep 2, 2022
@jtpio jtpio deleted the update-gitpod branch September 2, 2022 07:43
@jtpio
Copy link
Member Author

jtpio commented Sep 2, 2022

while there is this work being done to add Gitpod (and it would reduce the friction for Jupyterlab dev and contributions), I would like to suggest adding docs to the developer docs such as https://numpy.org/devdocs/dev/development_gitpod.html
I can confirm that adding Gitpod + documentation has been a game-changer for contributions, sprints and even maintainers workflows for NumPy and SciPy, so worth a shot.

Just checked the config on the numpy repo: https://github.com/numpy/numpy/blob/main/.gitpod.yml, and it looks very clean.

This seems to require building and pushing a custom Docker image in https://github.com/numpy/numpy/blob/main/.github/workflows/gitpod.yml. Probably this could also be made compatible with GitHub Codespaces then.

@jtpio jtpio mentioned this pull request Sep 5, 2022
4 tasks
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants