How to use poetry in Colab?

[elise-chin-adway]

How to use poetry in Colab?

Context

I am currently working on a team project, where we need to train neural networks. Some members are working on their local computer, and some on Colab (for GPU usage). We need to have the same dependencies. I am already familiar in using poetry on a local computer, but not on Colab, and I was wondering how to use it in Colab.

So I did some tests, and I encountered some issues. Maybe I can find some answers here. Thank you in advance! 😄

Issues

1st issue: poetry add <package> does not update pyproject.toml

I want to add a new package, suppose torch.
According to poetry's documentation, to install a new package, we need to run poetry add <package>. Since I run this command for the first time, a virtual environment is created, as well as the poetry.lock. But no packages were installed. Moreover, the poetry.lock file is updated, but not the pyproject.toml. This happens only on Colab. I tried on my local computer, and the command indeed automatically update the pyproject.toml file as well.

2nd issue: poetry run pip install <package> does not update pyproject.toml

Instead, we can install the package with poetry run pip install <package>. I saw this command on the following GitHub gist. The packages are now installed in the virtual environment, but the pyproject.toml was not updated.

Here is a link to the Colab notebook I used for those tests.

Thank you again!

[dimbleby]

https://gist.github.com/DPaletti/da7729c35ba7f9274c3635849608b7bc?permalink_comment_id=4455088#gistcomment-4455088

[elise-chin-adway]

Hello, thank you for your reply. I indeed saw this GitHub gist (I linked it in the 2nd issue description), and actually followed its instructions, but then encountered those issues.

[dimbleby]

I was linking to the comment in which someone told you that they had encountered the same problem and how they fixed it - did that help?

[elise-chin-adway]

Ah yes, I didn't see it! I just tried the command, and it works.
I also settled on another way to work with Colab.

Thanks!

[elise-chin-adway]

Hello for everyone reading this post.

I settled on a solution. The main issue was that pyproject.toml was not updated automatically, so I just decided to modify it by hand.
Here is the steps for using poetry in Colab, whether you create your own poetry project, or cloning a repo on Github.

https://github.com/elise-chin-adway/poetry-and-colab/blob/main/Using_python_poetry_in_Google_Colab.ipynb

I don't know if it is the best solution or not, but I hope it will help other people :)