Install nightly branches of official plugins when using Tutor Nightly

[kdmccormick]

Context

Installing Tutor "Stable", along with all the official plugins, involves one command:

pip install tutor[full]

Installing Tutor Nightly, on the other hand, involves some virtualenv and repository management:

cd $preferred_directory
workon tutorenv  # or, source venv/bin/activate, or whatever
git clone --branch=nightly https://github.com/overhangio/tutor.git
pip install -e ./tutor

plus an additional step for each plugin:

git clone --branch=nightly https://github.com/.../tutor-pluginX.git
pip install -e ./tutor-pluginX
tutor plugins enable pluginX

Would it be worth simplifying the installation process for Nightly?

Ideas (see issue comments for discussion)

[REJECTED] Allow Tutor Nightly to be installed from PyPI

I can think of a few different ways to do this:

1. Publish -nightly releases of tutor to PyPI alongside stable releases. By doing this, pip install tutor would still install Tutor Stable, but pip install --pre tutor would install Tutor Nightly.
2. Publish tutor-nightly to PyPI as an empty meta-package depending on the latest -nightly tutor release. This is an extension if idea 1 that would allow users to instead type pip install tutor-nightly.
3. Publish tutor-nightly to PyPI as a full package and expose a tutor binary. In this case, we would push nightly releases under a separate PyPI project name, with no dependency between the tutor and tutor-nightly packages. Unfortunately, we would have to account for users installing both tutor and tutor-nightly side-by-side. How would a system decide whether tutor refers to the Stable or Nightly installation? Well, since they would both expose the tutor=tutor.commands.cli:main console_scripts entrypoint, which setuptools handles by writing a script to $venv/bin, whichever package was installed more recently would "win".
4. Publish tutor-nightly to PyPI as a full package and expose a tutor-nightly binary. This is the same as idea 3, but it avoids the binary name conflict by just using a different name. Using tutor would always refer to Tutor Stable. I don't like this idea, but I'm including it for completeness.

[ACCEPTED] Enable one-line installation of official plugins in Tutor nightly

Official plugins are installed with Tutor Stable when the user types pip install tutor[full]. This doesn't work with Nightly because the extra requirements from [full] are constrained to the latest stable release of each official plugin.

I think it would be good if the full Nightly installation depended upon either -nightly releases of each plugin or even just a -e git+https://github.com/... pin. This change could be made on Tutor's nightly branch. Alternatively, if we went with the tutor-nightly meta-package idea, the Nightly plugin pins could be specified just in the meta-package's setup.py

Together with the Nightly PyPI Release idea, the installation process for a Nightly user would become:

pip install --pre tutor[full]

or:

pip install tutor-nightly[full]

Acceptance

Replace every line from plugins.txt in the nightly branch by -e git+https://github.com/overhangio/tutor-pluginname@nightly. Thus, running pip install -e ./tutor[full] would effectively install all plugins from the nightly branch.

The installation process would become:

cd $preferred_directory
workon tutorenv  # or, source venv/bin/activate, or whatever
git clone --branch=nightly https://github.com/overhangio/tutor.git ./tutor
pip install --upgrade -e ./tutor[full]

[kdmccormick]

@regisb This one's ready for your input now.

[regisb]

I agree that the installation procedure of tutor-nightly is too complex. In my opinion, the main source of complexity is the installation of plugins. I might be wrong, but I think that developers are ok with installing from a cloned repository, in most cases. But having to git clone every single plugin repo is a pain.

So I suggest the following:

git clone --branch=nightly https://github.com/overhangio/tutor.git ./tutor
pip install --upgrade -e ./tutor[full]

We would replace every line from plugins.txt in the nightly branch by -e git+https://github.com/overhangio/tutor-pluginname@nightly. Thus, running pip install -e ./tutor[full] would effectively install all plugins from the nightly branch.

Would that be acceptable?

I'm not so excited by the idea of publishing releases from the nightly branch to pypi. The main reason for that is that the nightly branch follows a "rolling release" pattern, where new changes can be pushed frequently to the nightly branch. Usually, when a release exists, it keeps working for some time. But in the case of the nightly branch, it would not, because edx-platform (and other IDAs) would keep changing and their master branches would no longer be current.

[kdmccormick]

git clone --branch=nightly https://github.com/overhangio/tutor.git ./tutor
pip install --upgrade -e ./tutor[full]

This looks great.

In my opinion, the main source of complexity is the installation of plugins. I might be wrong, but I think that developers are ok with installing from a cloned repository, in most cases.

True. And to your point, devstack always required a clone too. So, with plugin installation simplified, there won't be anything net-new here.

Usually, when a release exists, it keeps working for some time. But in the case of the nightly branch, it would not

I hadn't though about this but it makes sense. Let's go with your suggestion, and if we find there's still too much friction then we can reassess later.

[kdmccormick]

Fixed by overhangio/tutor#626, which is ready for review.