Build for Python 3 #356

jaedb · 2019-01-31T19:09:47Z

Upgrade all python code to 3, with backwards compatibility to 2.7 (which is what Mopidy currently supports).

UPDATE: The Iris Python3 version will not be backwards compatible to Python 2.7 (as is the case with Mopidy 3).

kingosticks · 2019-11-09T17:39:07Z

Mopidy 3.0.0a3 is now on PyPI, this pre-release removes support for Python 2.7 and runs under Python 3.7 and 3.8. See mopidy/mopidy#779 (comment)

jodal · 2019-12-15T19:03:48Z

We're targeting a Mopidy 3 release Dec 21. Do you think Iris will have a working pre-release out by then?

jaedb · 2019-12-15T19:39:12Z

Unfortunately that's not looking likely. This is my only (and first!) Python project so it will take me a bit of time to work through the upgrade. I have started but progress is slow at this stage.
Completion late January would be more realistic, also given the summer holiday period kicks off next week in New Zealand.

kingosticks · 2019-12-15T21:13:57Z

This porting checklist saves a lot of time, especially when you don't have much experience with Python 3 (like me).

kingosticks · 2019-12-20T09:48:52Z

Would it be helpful if I ran through the above checklist (as much as I can) and submitted a PR?

jaedb · 2019-12-21T02:05:17Z

@kingosticks I am grateful for any assistance, so yes please :-)

jaedb · 2019-12-21T03:01:02Z

I'm trying to install Mopidy 3.0.0-b1 but it's not available on pip for some reason, and running the mentioned python3 -m pip install --user Mopidy --pre installs version 2.1.
What's the best way to install the Python3 version of Mopidy?

 python3 -m pip install --user Mopidy==3.0.0b1
Collecting Mopidy==3.0.0b1
  Could not find a version that satisfies the requirement Mopidy==3.0.0b1 (from versions: 0.1.0a0, 0.1.0a1, 0.1.0a2, 0.1.0a3, 0.1.0, 0.2.0, 0.2.1, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.13.0, 0.14.0, 0.14.1, 0.14.2, 0.15.0, 0.16.0, 0.16.1, 0.17.0, 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.19.2, 0.19.3, 0.19.4, 0.19.5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.1.0, 1.1.1, 1.1.2, 2.0.0, 2.0.1, 2.1.0)
No matching distribution found for Mopidy==3.0.0b1

kingosticks · 2019-12-21T12:40:40Z

I just installed it into a fresh python3.7 virtualenv using python3 -m pip install --user Mopidy==3.0.0b1. What version of Python 3 do you have here?

You can also install from source by following the instructions at https://docs.mopidy.com/en/develop/devenv/

jaedb · 2019-12-22T01:11:29Z

Hmmm seems peculiar. Ideally I'd run this in Docker but that can wait until I get the basics sorted.

python3 --version
Python 3.6.9

kingosticks · 2019-12-22T02:48:10Z

Mopidy 3 requires Python 3.7

jaedb · 2019-12-23T06:48:35Z

Mopidy should work on 3.7.6 then right?

root@d08d97de611f:/# mopidy

ERROR: A GObject based library was not found.

Mopidy requires GStreamer to work. GStreamer is a C library with a
number of dependencies itself, and cannot be installed with the regular
Python tools like pip.

Please see http://docs.mopidy.com/en/latest/installation/ for
instructions on how to install the required dependencies.

Traceback (most recent call last):
  File "/usr/local/bin/mopidy", line 5, in <module>
    from mopidy.__main__ import main
  File "/usr/local/lib/python3.7/site-packages/mopidy/__main__.py", line 7, in <module>
    from mopidy import commands
  File "/usr/local/lib/python3.7/site-packages/mopidy/commands.py", line 14, in <module>
    from mopidy.audio import Audio
  File "/usr/local/lib/python3.7/site-packages/mopidy/audio/__init__.py", line 2, in <module>
    from .actor import Audio
  File "/usr/local/lib/python3.7/site-packages/mopidy/audio/actor.py", line 8, in <module>
    from mopidy.audio import tags as tags_lib
  File "/usr/local/lib/python3.7/site-packages/mopidy/audio/tags.py", line 7, in <module>
    from mopidy.internal.gi import GLib, Gst
  File "/usr/local/lib/python3.7/site-packages/mopidy/internal/gi.py", line 5, in <module>
    import gi
ModuleNotFoundError: No module named 'gi'

kingosticks · 2019-12-23T10:12:02Z

Yes it should.

Since there's no .deb out yet, the idea is to follow https://docs.mopidy.com/en/latest/installation/pypi/.

However, if you are on Ubuntu 18.04, or something else that only ships with Python 3.6 as default, then it's a little more complicated; each Ubuntu release has a python3-gst-1.0 which is compiled with the system Python only. In which case you need to acquire a version of python3-gst-1.0 designed to work with Python 3.7 (or 3.8 if that's your target Python). For my Ubuntu 18.04 machine I ended up grabbing the cosmic version from https://packages.ubuntu.com/cosmic/python3-gst-1.0 but that's very hacky. We need better instructions for people in this situation @jodal .

But if you are working with docker you could just use our image?

fmarzocca · 2019-12-25T11:02:24Z

I suppose I have to wait for Iris migration to pyhton3 before installing mopidy 3, correct?

kingosticks · 2019-12-25T11:56:25Z

Yes.

…

On Wed, 25 Dec 2019, 11:02 Fabio Marzocca, ***@***.***> wrote: I suppose I have to wait for Iris migration to pyhton3 before installing mopidy 3, correct? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#356?email_source=notifications&email_token=AAHEHKDQ2D7A7KEHAVCLF6DQ2M4UBA5CNFSM4GTT2UNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHUIGMA#issuecomment-568886064>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHEHKBXWTLQWPAV6VTIPCLQ2M4UBANCNFSM4GTT2UNA> .

MarcinWieczorek · 2019-12-30T21:02:14Z

Hello.

My current notes for Iris on mopidy 3 are:

This is useful

https://mopidy.com/blog/2019/12/27/mopidy-3-faq/

`ModuleNotFoundError: No module named 'gi'`

is solved easily in above page with pip install --ignore-installed --no-cache pygobject

Tornado is not compatible

I was still using outdated tornado to get Iris working and the issue is still valid.
jupyter/notebook#3397 (comment)
hotfix: pip3 install tornado==4.5.3 (requires real fix)

Urlencode fix

Too little to create a pull request :)

From 9648c5cb8c0d5b20fc9f2de53ed66658566aaab6 Mon Sep 17 00:00:00 2001
From: Marcin Wieczorek <marcin@marcin.co>
Date: Mon, 30 Dec 2019 21:57:35 +0100
Subject: [PATCH] Fix urlencode

---
 mopidy_iris/core.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mopidy_iris/core.py b/mopidy_iris/core.py
index d9b21602..33b2227a 100755
--- a/mopidy_iris/core.py
+++ b/mopidy_iris/core.py
@@ -988,7 +988,7 @@ class IrisCore(pykka.ThreadingActor):
 
         try:
             http_client = tornado.httpclient.HTTPClient()
-            request = tornado.httpclient.HTTPRequest(url, method='POST', body=urllib.urlencode(data))
+            request = tornado.httpclient.HTTPRequest(url, method='POST', body=urllib.parse.urlencode(data))
             response = http_client.fetch(request)
 
             token = json.loads(response.body)
-- 
2.24.1

UI issue with notification:

Create process notification works, but cannot be canceled.

`Run test process` does not work, even blocking

I don't know what is the sudo for, but the fallback way doesn't seem to work.

Soooo after what I've wrote above it plays music!

jaedb · 2019-12-31T00:43:50Z

@MarcinWieczorek that's awesome, thanks heaps! Is that based off the current state of the feature/python3 branch?

MarcinWieczorek · 2019-12-31T00:45:56Z

Yes indeed

jaedb · 2020-01-25T09:19:34Z

Keen for your thoughts on 564839c. Essentially Frontend uses IrisCore's ioloop that is provided by the Iris websocket in handlers.py.

fmarzocca · 2020-01-30T08:33:08Z

I have seen there is a new release. Is this issue fixed?

fmauNeko · 2020-01-30T08:45:44Z

The changelog says

Support for Mopidy 3 / python 3.8
No backwards compatibility to Mopidy 2 / python 2.7

So I guess that must indeed be solved.
The package isn't released on PyPI yet though.

fmarzocca · 2020-01-30T08:51:56Z

The package isn't released on PyPI yet though.

thanks.
How can I get informed about pip3 release of Iris? Will it be also added to mopidy's apt repository?

tgurr · 2020-01-30T12:07:50Z

I'd also welcome a release on PyPI so we can update our distribution package to the latest Iris version.

@jaedb One thing I noticed in the tarball from Github is that setup.cfg has version = 3.0.0 shouldn't it state 3.44.0 instead? And there's also a file VERSION included which has 3.7.5-2 in it.

Gadgetoid · 2020-01-30T12:21:09Z

Thank you for getting this updated so quickly @jaedb - I can't imagine a Mopidy without the Iris front-end. Eagerly awaiting a pip release!

MarcinWieczorek · 2020-01-30T12:30:06Z

I'm glad to see the update. @jaedb, has anything changed about the dependendies? I'm about to update the package in AUR.

sandyjmacdonald · 2020-01-31T12:33:06Z

Nice work, @jaedb! It's working well for me. Also eagerly awaiting a pip release :-)

JayGatsby7 · 2020-01-31T15:15:50Z

Nice work, @jaedb! It's working well for me. Also eagerly awaiting a pip release :-)

Wouldn’t mind seeing some idiot proof instructions for how to install this on a raspberry pi using docker. I have zero experience with docker but like the idea of it from a security standpoint.

MarcinWieczorek · 2020-01-31T15:16:57Z

Nice work, @jaedb! It's working well for me. Also eagerly awaiting a pip release :-)

Wouldn’t mind seeing some idiot proof instructions for how to install this on a raspberry pi using docker. I have zero experience with docker but like the idea of it from a security standpoint.

pip install mopidy-whatever in dockerfile
https://github.com/wernight/docker-mopidy

JayGatsby7 · 2020-02-01T00:14:47Z

Ahh so the docker container is mopidy. Now I see

JayGatsby7 · 2020-02-01T00:36:13Z

That doesn’t seem practical on a raspberry pi

MarcinWieczorek · 2020-02-02T18:37:58Z

That doesn’t seem practical on a raspberry pi

It's as practical as it gets. You can have mopidy separated from python packages which is good (I think I'm going to switch to that soon). It's similar to just using a virtualenv, but with docker.

jaedb · 2020-02-03T02:28:56Z

Iris does have a Docker container and will be uploaded to Docker Hub when the final stages of release are finished.

jaedb · 2020-02-03T02:56:21Z

@kingosticks do you have any advice on the check_manifest tox test? The /src path doesn't need to be in the distribution (but it does need to be in VCS) and I can't see how it can be omitted without breaking the test.

BrainStone · 2020-02-03T10:56:29Z

Is there a somewhat working test version I could try at the moment?

jodal · 2020-02-03T14:31:13Z

@kingosticks do you have any advice on the check_manifest tox test? The /src path doesn't need to be in the distribution (but it does need to be in VCS) and I can't see how it can be omitted without breaking the test.

You can ignore src/ in a check-manifest section in pyproject.toml, ref. https://github.com/mgedmin/check-manifest#configuration

kingosticks · 2020-02-03T14:41:25Z

But isn't that already specified at https://github.com/jaedb/Iris/blob/master/tox.ini#L28 ?

Is this because it should be src/** in there (are they even globs?).
EDIT: they are not globs, it uses fnmatch.

jaedb · 2020-02-03T18:15:19Z

@BrainStone yes, you should be able install from source using the develop branch:

Clone this repo
Checkout develop branch
Run python3 -m setup.py install

BrainStone · 2020-02-04T08:06:09Z

@jaedb would something like python3 -m pip install https://github.com/jaedb/Iris/archive/develop.zip work too?

Edit: Yes it does.

brammittendorff · 2020-02-07T12:32:15Z

When using the following packages:

Mopidy==3.0.1
Pykka==2.0.2
Mopidy-Local==3.1.1
Mopidy-AlsaMixer==2.0.0
Mopidy-Spotify==4.0.1
git+https://github.com/jaedb/Iris.git@develop
Mopidy-SoundCloud==3.0.0
Mopidy-GMusic==4.0.0
Mopidy-Youtube==3.0a1

Working:

Youtube
Local
AlsaMixer
Soundcloud
GMusic

Not working

Spotify

After searching with Spotify i get this error:

ERROR    2020-02-07 13:29:52,322 [418:HttpServer] tornado.application
  Uncaught exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/tornado/http1connection.py", line 238, in _read_message
    delegate.finish()
  File "/usr/local/lib/python3.7/dist-packages/tornado/routing.py", line 251, in finish
    self.delegate.finish()
  File "/usr/local/lib/python3.7/dist-packages/tornado/web.py", line 2097, in finish
    self.execute()
  File "/usr/local/lib/python3.7/dist-packages/tornado/web.py", line 2117, in execute
    **self.handler_kwargs)
2020/02/07 13:29:52 [error] 542#542: *15 upstream prematurely closed connection while reading response header from upstream, client: 172.30.32.2, server: local-mopidy, request: "GET /iris/http/refresh_spotify_token HTTP/1.1", upstream: "http://127.0.0.1:4478/iris/http/refresh_spotify_token", host: "fred.local", referrer: "http://fred.local/iris/search/all/test"
  File "/usr/local/lib/python3.7/dist-packages/tornado/web.py", line 190, in __init__
    self.clear()
  File "/usr/local/lib/python3.7/dist-packages/tornado/web.py", line 294, in clear
    self.set_default_headers()
  File "/usr/local/lib/python3.7/dist-packages/mopidy_iris/handlers.py", line 183, in set_default_headers
    Authorization, Client-Security-Token, Accept-Encoding''',
  File "/usr/local/lib/python3.7/dist-packages/tornado/web.py", line 339, in set_header
[07/Feb/2020:13:29:52 +0100] 502 10.53.8.94, 172.30.33.6, 172.30.32.1(172.30.32.2) GET /iris/http/refresh_spotify_token HTTP/1.1 (Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/79.0.3945.79 Chrome/79.0.3945.79 Safari/537.36)
    self._headers[name] = self._convert_header_value(value)
  File "/usr/local/lib/python3.7/dist-packages/tornado/web.py", line 388, in _convert_header_value
    raise ValueError("Unsafe header value %r", retval)
ValueError: ('Unsafe header value %r', 'Origin, X-Requested-With, Content-Type, Accept,\n            Authorization, Client-Security-Token, Accept-Encoding')

Even with tornado-4.5.3:

Successfully built tornado
Installing collected packages: tornado
  Attempting uninstall: tornado
    Found existing installation: tornado 6.0.3
    Uninstalling tornado-6.0.3:
      Successfully uninstalled tornado-6.0.3
Successfully installed tornado-4.5.3

jjok · 2020-02-07T12:36:14Z

@brammittendorff I had that problem too, but then I installed https://github.com/jaedb/Iris/archive/3.44.0.zip instead of develop.zip and it worked.

brammittendorff · 2020-02-07T12:45:41Z

@jjok You are right that one works, even with Spotify thanks alot. So the working setup is:

Mopidy==3.0.1
Pykka==2.0.2
Mopidy-Local==3.1.1
Mopidy-AlsaMixer==2.0.0
Mopidy-Spotify==4.0.1
git+https://github.com/jaedb/Iris.git@3.44.0
Mopidy-SoundCloud==3.0.0
Mopidy-GMusic==4.0.0
Mopidy-Youtube==3.0a1

jaedb · 2020-02-07T20:00:12Z

This has just been released to pip and the image to Docker hub so I shall close this issue. Please open a new issue for any problems you may encounter.

fmarzocca · 2020-02-07T20:40:47Z

I have found only v.3.43.0 in pip3.

JayGatsby7 · 2020-02-08T01:14:44Z

I have found only v.3.43.0 in pip3.

sudo python3 -m pip install Mopidy-Iris [should work for you]

jaedb added the P2 Medium priority (important) label Jan 31, 2019

jaedb added the help wanted label Feb 8, 2019

jaedb changed the title ~~Build for Python 2.7 and 3~~ Build for Python 3 Nov 18, 2019

jaedb added in progress P1 High priority (next release) and removed P2 Medium priority (important) help wanted labels Nov 18, 2019

jaedb pinned this issue Nov 19, 2019

jjok mentioned this issue Dec 5, 2019

Upgrade to Mopidy 4 / Python 3 jjok/my-mopidy-setup#4

Closed

Gadgetoid mentioned this issue Dec 31, 2019

mopidy 3.x breaks installation pimoroni/pirate-audio#9

Closed

docmollo mentioned this issue Jan 3, 2020

Lock Install to Python 2.x pimoroni/pirate-audio#10

Merged

jaedb mentioned this issue Jan 4, 2020

Import paths wrong? No module named 'handlers' #474

Closed

mikebarkmin mentioned this issue Jan 8, 2020

Fix spotify mopidy3 for buster MiczFlor/RPi-Jukebox-RFID#701

Merged

veloxidSchweiz mentioned this issue Feb 5, 2020

Install mopidy-2.2.2-1 from main repository MiczFlor/RPi-Jukebox-RFID#771

Merged

jaedb closed this as completed Feb 7, 2020

jaedb removed the in progress label Feb 10, 2020

jaedb unpinned this issue Feb 10, 2020

Build for Python 3 #356

Build for Python 3 #356

Comments

jaedb commented Jan 31, 2019 • edited Loading

kingosticks commented Nov 9, 2019

jodal commented Dec 15, 2019

jaedb commented Dec 15, 2019

kingosticks commented Dec 15, 2019

kingosticks commented Dec 20, 2019

jaedb commented Dec 21, 2019

jaedb commented Dec 21, 2019

kingosticks commented Dec 21, 2019 • edited Loading

jaedb commented Dec 22, 2019

kingosticks commented Dec 22, 2019

jaedb commented Dec 23, 2019

kingosticks commented Dec 23, 2019 • edited Loading

fmarzocca commented Dec 25, 2019

kingosticks commented Dec 25, 2019 via email

MarcinWieczorek commented Dec 30, 2019

This is useful

ModuleNotFoundError: No module named 'gi'

Tornado is not compatible

Urlencode fix

UI issue with notification:

Run test process does not work, even blocking

jaedb commented Dec 31, 2019

MarcinWieczorek commented Dec 31, 2019 via email • edited Loading

jaedb commented Jan 25, 2020 • edited Loading

fmarzocca commented Jan 30, 2020

fmauNeko commented Jan 30, 2020

fmarzocca commented Jan 30, 2020

tgurr commented Jan 30, 2020

Gadgetoid commented Jan 30, 2020

MarcinWieczorek commented Jan 30, 2020

sandyjmacdonald commented Jan 31, 2020

JayGatsby7 commented Jan 31, 2020

MarcinWieczorek commented Jan 31, 2020

JayGatsby7 commented Feb 1, 2020

JayGatsby7 commented Feb 1, 2020

MarcinWieczorek commented Feb 2, 2020

jaedb commented Feb 3, 2020

jaedb commented Feb 3, 2020 • edited Loading

BrainStone commented Feb 3, 2020

jodal commented Feb 3, 2020

kingosticks commented Feb 3, 2020 • edited Loading

jaedb commented Feb 3, 2020

BrainStone commented Feb 4, 2020 • edited Loading

brammittendorff commented Feb 7, 2020 • edited Loading

jjok commented Feb 7, 2020

brammittendorff commented Feb 7, 2020 • edited Loading

jaedb commented Feb 7, 2020

fmarzocca commented Feb 7, 2020

JayGatsby7 commented Feb 8, 2020

jaedb commented Jan 31, 2019 •

edited

Loading

kingosticks commented Dec 21, 2019 •

edited

Loading

kingosticks commented Dec 23, 2019 •

edited

Loading

`ModuleNotFoundError: No module named 'gi'`

`Run test process` does not work, even blocking

MarcinWieczorek commented Dec 31, 2019 via email •

edited

Loading

jaedb commented Jan 25, 2020 •

edited

Loading

jaedb commented Feb 3, 2020 •

edited

Loading

kingosticks commented Feb 3, 2020 •

edited

Loading

BrainStone commented Feb 4, 2020 •

edited

Loading

brammittendorff commented Feb 7, 2020 •

edited

Loading

brammittendorff commented Feb 7, 2020 •

edited

Loading