Skip to content

Moving kivy.garden.xxx to kivy_garden.xxx and kivy.deps.xxx to kivy_deps.xxx

Jump to bottom
matham edited this page May 21, 2019 · 4 revisions

Kivy garden

When we originally developed the garden concept, where people can host their widgets (flowers) that other people can download, we also developed the kivy-garden tool that lets one install these flowers. The structure of these flowers were specified such that the garden tool can simply download the files and then kivy, through a import hook, would be able to use these flowers. This meant we wouldn't use the typical python packaging mechanism (setup.py, proper package structure).

However, there are many issues with this approach that we discovered over the years, such as no cython support (or anything that requires setup), having to teach users how to use a new custom tool, and maintenance issues where we don't have the resources to deal with bugs that come up. Additionally, by default installing packages outside typical python hierarchies (in the user home folder), we circumvent any virtual environment or environment localization. Finally, there's no easy way to specify a complete installation with a requirements.txt type file.

Proposal

I have proposed a new approach to garden that uses normal python packaging with pip and have provided a detailed migration path and some demo packages to make the migration easier. Specifically, here's how garden packages would appear:

  • We will have a proper kivy_garden namespace where garden packages will be installed to. So e.g. if you want to install the graph flower, you'd do pip install kivy_garden.graph and it would install the package under kivy_garden/graph. This would be external to kivy and part of the python environment like any normal python package.
  • Garden packages would have a normal setup.py and complete python package structure. I wrote a demo pure-python and cython-python flower that should be used for new packages and for migrating old packages.
  • Instead of the garden tool, users will have a choice to either
    • install directly from pypi, e.g. pip install kivy_garden.graph.
      • kivy will not make automated releases to pypi, it would be up to contributors to do that. But we do have guides for that in the flower demo.
    • Install directly from github pip install https://github.com/kivy-garden/graph/archive/master.zip or pip install https://github.com/kivy-garden/graph/archive/some_release.zip.
    • Install using kivy's simple pypi server that we host on github: pip install kivy_garden.graph --extra-index-url https://kivy-garden.github.io/simple/. This server is hosted on github. To add new flowers people will be making PRs. Similarly, to release new versions, add pre-compiled wheels stored on github release assets, users will be making PRs to that repo. Eventually we can automate this, but for now it's easy to make PRs to add new packages or new releases to existing packages.
  • Packaging with pyinstaller will work as with normal python packages.

The legacy garden packages and the garden tool will be able to be installed side by side with the newer packages because they are installed to different places.

Kivy deps

We invented the kivy.deps.xxx packages to be able to ship kivy in wheel form. We needed a way to ship the dependencies so we decided to store the deps under kivy itself in kivy/deps/xxx because that seemed the best option at the time.

However, as it turns out this has some issues that currently cannot be overcome without manual copying of files. Specifically, if kivy is installed as editable and we install a dependency, pip is unable to install it at the editable location and instead installs it under site-packages. This breaks kivy and the dependency has to be manually copied to the editable location.

Also, installing dependencies directly under kivy is not a great choice from an isolation POV. And by doing this we are not using the typical form of namepsace packages.

Instead, like kivy_garden, I am proposing that we add support for the kivy_deps name spaces with packages such as kivy_deps.sdl2. These packages would be installed in kivy_deps/xxx, outside the kivy hierarchy. Here's the PR for this proposal.

Migration process

pip does not distinguish between packages named kivy.deps.xxx and kivy_deps.xxx because to pip, both are converted to kivy-deps-xxx. What this means is that pip and pypi consider kivy.deps.xxx and kivy_deps.xxx as the same project and if you upload them they are stored in the same project on pypi. Similarly, no matter which you request with pip install ... pip will choose the one with the latest version specifier whether it is kivy.deps.xxx or kivy_deps.xxx. This is one of the reasons why we should move away from the kivy/deps/xxx hierarchy as that leads to more conflicts.

Even though pip does not differentiate between kivy.deps.xxx and kivy_deps.xxx, they still are installed to the proper locations (kivy.deps.xxx to kivy/deps/xxx and kivy_deps.xxx to kivy_deps/xxx).

So I propose that we release all the kivy_deps.xxx packages such that their version is newer than the kivy.deps.xxx packages and that we deprecate the kivy.deps.xxx packages so that their version will not increase. This is easily done. What this is means is as follows:

  • If you are on release 1.11.0 (assuming this is merged before that release) or kivy master, you should just install packages with kivy_deps.xxx, this will install them to kivy_deps/xxx and everything will work.
  • If you are on a older kivy and don't plan to upgrade, you should pin all your kivy.deps.xxx to specific versions that were released at the time of your kivy release. This will prevent the newer kivy_deps.xxx from being installed. You will need to do this anyway because the latest gstreamer may cause issues for older kivy version before 1.11.0. And we should get our users in the habit of pinning their versions anyway.
  • If you are migrating to 1.11.0, you should uninstall all your deps with pip, or make sure they are upgraded with pip (pip install --upgrade kivy_deps...) when updating to 1.110 or newer. This will give you the kivy_deps/xxx versions.
Clone this wiki locally