Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to Python 3 #779

Open
jodal opened this issue Jul 12, 2014 · 21 comments

Comments

@jodal
Copy link
Member

commented Jul 12, 2014

Python 3 is the future of Python, it is widely available on the environments Mopidy is deployed on, and I'm quite interested in being able to use asyncio, which requires Python 3.3+. There's nothing outside the Mopidy project blocking us from moving to Python 3.

This is the master bug to track our way towards running Mopidy on Python 3.

  • Dependencies:
    • Pykka supports Python 2.6+, 3.2+.
    • Tornado supports Python 2.6+, 3.2+.
    • GStreamer:
      • #225: Switch GStreamer Python bindings from gst-python (Python 2 only) to PyGI (Python 2+3). To get everything working with PyGI, a port of GStreamer from 0.10 to 1.x at the same time is required.
      • #665: Make mixers independent of GStreamer, as mixers are removed in GStreamer 1.x.
      • #671: Extract/reimplement our playlist format support to not depend on GStreamer. There's some issues with reimplementing this on top of PyGI/GStreamer 1.x.
  • Move code style towards being Python 2+3 compatible:
    • flake8 warning clean code.
    • Drop support for all Python versions older than 2.7.
    • Use from __future__ import unicode_literals everywhere and mark binary strings with b'foo'.
    • Selectively fix portability issues identified by the 2to3 tool.
  • Development support:
    • Use tox to run tests, so we can easily add Python 3 testing in parallel with Python 2.7.
  • Mopidy-Spotify:
    • Implement new libspotify binding, pyspotify 2, which supports Python 2.7, 3.2+.
    • Reimplement Mopidy-Spotify on top of pyspotify 2.
  • Mopidy-MPRIS:
    • python-indicate bindings to libindicate can be replaced with PyGI.
    • python-dbus can maybe be replaced by GDBus through PyGI.
  • Extension ecosystem:
    • Use tox as test runner in cookiecutter-mopidy-ext to be ready to start running tests on Python 3 too.
    • Send pull requests/create issues for Python 2+3 support in all extensions when Mopidy itself works on Python 2+3.
  • Once ~all extensions supports Python 2+3, drop Python 2 support in Mopidy.
  • Remove Python 2isms.
  • Enjoy new Python 3 features, like asyncio.

@jodal jodal added the enhancement label Jul 12, 2014

@jodal jodal self-assigned this Jul 12, 2014

@jodal jodal added this to the v0.22 - PyGI and GStreamer 1.x milestone Feb 5, 2015

@DanielFallon

This comment has been minimized.

Copy link

commented Feb 19, 2016

Hey, I'd like to help with Python 3 migration, anything in particular that's under way? don't want to step on anyone's toes

@jodal

This comment has been minimized.

Copy link
Member Author

commented Feb 19, 2016

I've started a branch at jodal/feature/py3-compat where I've:

  • updated the Python version check that stops you from starting Mopidy under Python 3
  • updated the tox.ini setup
  • starting fixing Py3 issues module-by-module, running the tests under both Python 2 and Python 3

I haven't touched this since November, except that I rebased the branch on top of Mopidy 2.0.0 a couple of days ago. The current status is that I've fixed around 1000 tests, with about 240 remaining.

The plan ahead is to:

  1. Fix the rest of the issues module-by-module.
  2. Take the experience from fixing all the issues and try to land as much of it as possible in Mopidy 2.x. My current module-by-module approach changes things (e.g. it starts to threat file paths as Unicode instead of bytes) that I don't want to land in Mopidy until the 3.0 release, where we're allowed to break things. The can't-land-until-3.0 diff should be as small as possible.
  3. Use Mopidy with Python 3, finding all the issues not covered by tests.
@DanielFallon

This comment has been minimized.

Copy link

commented Feb 19, 2016

ok, I will fork off of that branch and try to fix whatever tests I can

@aleprovencio

This comment has been minimized.

Copy link

commented Dec 10, 2016

Hello guys, any news on this issue?

@flying-sheep

This comment has been minimized.

Copy link

commented Dec 10, 2016

3½ years left for this 😉

@Prior99

This comment has been minimized.

Copy link

commented Mar 22, 2017

Are there any updates on this?

@jodal

This comment has been minimized.

Copy link
Member Author

commented Mar 22, 2017

Porting to Python 3 is my personal highest priority task when working on Mopidy, problem is I have been quite busy with work for the last year.

My current plan, in approximate order, is to:

  • Fix the Mopidy-MPRIS test suite, so that it can survive the upcoming changes. This is a bit boring and my doorstep mile.
  • Remove all the deprecated stuff from Mopidy and make a major release. See this milestone for the detailed issues. This I look forward to.
  • Ensure that all extensions in the Mopidy org still works after the removal, and make necessary changes and releases.
  • Port Mopidy to Python 2 + 3 and make another major release.
  • Port all extensions in the Mopidy org to Python 3 only and release.
  • Port/help port all other extensions to Python 3 only and urge for release.
  • Remove the Python 2 support from Mopidy itself.

As you can see, there's a couple of tasks that should be done first before starting the porting itself. That said, the largest body of work is already complete: Rewriting pyspotify to workon Python 3 (two years of my spare time was spent here) and porting Mopidy to GStreamer 1.x (one paternity leave of spare time spent here).

I'll get there, but it takes time. If people want to help out, I think the most helpful would be helping port extensions to Python 3 as soon as Mopidy supports 2+3.

@Prior99

This comment has been minimized.

Copy link

commented Mar 22, 2017

Thanks for the update, @jodal.

@eamanu

This comment has been minimized.

Copy link

commented Mar 23, 2017

@jodal

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2017

Any help with the project that frees some of the maintenance burden helps with the big hairy goals like Python 3, as it frees time from the maintainers to either work on the larger issues themselves or to do proper code review on larger contributes, which can be quite exhausting and time consuming. Two prime examples are helping other users with debugging their problems, and to reproduce possible bugs, minimize the steps needed to reproduce, and document the results in the issues.

@eamanu

This comment has been minimized.

Copy link

commented Mar 24, 2017

@jodal Where we can see your work with python3?

regards

@Equidamoid

This comment has been minimized.

Copy link

commented Dec 30, 2017

@jodal the py3-compat branch you mentioned looks a bit... abandoned. Is there a new one somewhere?
I'm considering writing an extension and I'd prefer digging into some py3 bugs in unstable mopidy branch rather than remembering nasty py2 workarounds.

@jaedb jaedb referenced this issue Mar 25, 2018

Closed

UI Not Updating after action #265

6 of 18 tasks complete
@hrehfeld

This comment has been minimized.

Copy link

commented Aug 20, 2018

What's the status on this? is there a version of mopidy that can be used with python3?

@hugovk

This comment has been minimized.

Copy link

commented Aug 20, 2018

Dependency pylast will be dropping support for legacy Python 2 soon: pylast/pylast#265

@hugovk hugovk referenced this issue Sep 8, 2018

Closed

Plan to drop support for Python 2.7 #265

12 of 12 tasks complete

@jodal jodal modified the milestones: v4.0, v3.0 Sep 29, 2018

@noahfx

This comment has been minimized.

Copy link

commented Jan 3, 2019

Hi,
Currently I maintain pylast in Debian. I intend to upload the latest version (1) which no longer supports Python 2.7
I plan to publish this version at the end of February, if mopidy still uses Python 2 will not be included
in the next Debian release which could be a shame since mopidy is an amazing piece of software 😃

Is there any official branch where this is going on? I am wiling to help this to happen

Regards

@jjok

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

@jodal Are the extensions the blocker here? If so, I would suggest opening an issue against each extension and referencing them here to keep track. If you can give some instructions in the issue on what changes are likely to be required, then people how aren't that familiar with Python (like me) could hopefully still help.

@jjok

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

PS I'm happy to help with the extentions that I use (GMusic, Iris, Podcast, Scrobbler, Spotify), I just don't really know what to do at the moment.

@sashko

This comment has been minimized.

Copy link

commented Feb 4, 2019

Hey folks, may someone please clarify the status of adaptation to python3? Do we have any estimates when the first python3 version will be released? Thanks.

@jodal

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

I'm still committed to porting Mopidy to Python 3. I won't let Mopidy die with Python 2.

History

Most of the work of moving us to Python 3 was done years ago in assuring that all our dependencies works on Python 3:

  • Pykka has been Python 3 compatible since it was extracted from Mopidy in 2011.
  • pyspotify 2.x was a full rewrite to get Python 3 support and complete libspotify bindings. This alone took almost two years from start to finish, work that is now mostly void thanks to Spotify breaking large parts of libspotify.
  • The port of Mopidy from GStreamer 0.10/PyGObject to GStreamer 1.x/PyGI, released in Mopidy 2.0 three years ago, was the primary project of my second paternity leave.

As you can probably read between the lines from the long history of this issue and the state of the issue tracker, this project and open source in general, hasn't been my top priority since I moved from working in a large enterprise to a startup three years ago.

However, I'm slowly starting to pick up my open source projects again. I'm trying to focus on one project at the time and ship it before taking on the next. So far Mopidy.js and Mopidy-MPRIS has received some tender love and care. Currently, I'm working on a new release of Pykka, which among other small improvements will reduce the startup time off many Mopidy setups by a huge factor.

Next up

Once Pykka 2 has shipped, my primary goal is moving Mopidy to Python 3.

I won't be giving any time estimate for when that work will be complete, as that would simply be setting myself up for failure. It all depends a lot on external factors and motivation.

As for helping out, I don't see that there's much new contributors can do to significantly help the effort directly. I'm still not back to working on open source in a predictable and regular cadence, so PR reviews would quickly become demotivating for both parties. As previously mentioned in this issue, any help to reduce the general maintenance burden always helps indirectly.

As for other core devs, I'm sure they would be able to help out, but we all have our priorities and projects. I won't wait for our time and motivation to all align up. If they join in once I get properly started on Mopidy 3, that would be lovely, but I have no expectations.

Putting all my rambling about life and everything aside, this is neither a gigantic task nor a task I'm unfamiliar with. It's mostly a matter of completing the other ongoing stuff so that this can become the primary focus for a while.

Mopidy 3

Since I wrote the last plan here in March 2017, Python 3 adoption has gotten so far that I no longer see the point in taking gradual steps from Python 2 only via 2+3 to Python 3 only. Mopidy 2.x has served us quite well for three years, and will be a good and stable place to leave those stuck on Python 2 for whatever reason.

Thus, revising the plan from March 2017, I imagine a process like this:

  • Remove all the deprecated stuff from Mopidy. See the v3.0 milestone for the detailed issues.
  • Port Mopidy to Python 3.
  • Make a pre-release to PyPI, e.g. 3.0.0-rc1. This won't affect those running pip install mopidy, but makes it possible to run pip install --pre mopidy to install the pre-release for testing of ported extensions.
  • Port all extensions in the Mopidy org to Python 3, and optionally make pre-releases to PyPI.
  • Port/help port other extensions to Python 3. Help wanted once we get here.
  • Make a coordinated major release of Mopidy core and all extensions in the Mopidy org.
@jodal

This comment has been minimized.

Copy link
Member Author

commented Jun 22, 2019

Since there has been five months, I guess it is time to update this issue. The Pykka release I spoke of last time was shipped as Pykka 2.0 six weeks ago. Since then, I've started working on Mopidy again:

  • Bugfix release 2.2.3 is out, so we have currently nothing unreleased in the release-2.2 branch.
  • In the develop branch which will become Mopidy 3.0 most of the deprecated stuff has been removed.

Next up for me is probably:

  • Simplifying logging (#1452)
  • Decide and implement how to handle file system paths when moving from byte-oriented Python 2 APIs to text-oriented Python 3 file system APIs. (No issue tracking this yet.)
  • Start making the test suite pass on Python 3.
@tmccombs

This comment has been minimized.

Copy link

commented Aug 17, 2019

Is there anything I could do to help with making either mopidy core, or the mopidy-soundcloud plugin work with python3?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.