🐛 | Connectivity issues with mopidy

[drybx]

Hi!

My phoniebox worked quite well and I was ready to hand it on to my godchild, but now - seemingly out of nowhere - new Spotify problems emerged:
I installed the latest developer version of phoniebox and ran the commands of #852 (even though I think they are all part of the developer script already).

Unfortunately I can't play any Spotify content even though it says "Connected" under server status (I do get search results though).

journalctl -u mopidy gives me this where the following lines stick out to me:
Apr 12 12:38:11 raspberrypi mopidy[465]: ERROR [SpotifyEventLoop] spotify.session Spotify login error: <ErrorType.UNABLE_TO_CONTACT_SERVER: 8> Apr 12 12:38:12 raspberrypi mopidy[465]: INFO [SpotifyBackend-6] mopidy_spotify.web Logged into Spotify Web API as joseiger Apr 12 13:24:34 raspberrypi mopidy[465]: INFO [SpotifyBackend-6] mopidy_spotify.lookup Failed to lookup 'spotify:track:0d6bZwEAi1XblGRhnzPfgC': Session must be logged in

So there seems to be some kind of connectivity issue even though it says Logged into Spotify Web API at some point.
The login information is definitely correct and I just created a new cliend ID and client secret.
I don't think there is a general problem with my internet, I don't use any proxy, VPN etc...

sudo systemctl status mopidy doesn't reveal any problems.

Any ideas whta to try?
If it helps, here ist the output of tail -n 500 /var/log/syslog.

Julius

[s-martin]

Could be a temporary error on Spotify side, don’t know if you can check on Spotify, if there are any issues currently.

I read here this error was also caused by expired credit card info on Spotify side:
https://discourse.mopidy.com/t/spotify-login-error-errortype-unable-to-contact-server-8/1400

[s-martin]

I installed the latest developer version of phoniebox and ran the commands of #852 (even though I think they are all part of the developer script already).

That’s correct, you only need #852 for master, not develop.

[drybx]

I read here this error was also caused by expired credit card info on Spotify side:
https://discourse.mopidy.com/t/spotify-login-error-errortype-unable-to-contact-server-8/1400

I paid my spotify bills via Paypal, switched to credit card now, which didn't change anything.
I also tried to switch to a different (paid) spotify account.

Earlier today, all of a sudden it worked and I had no idea why. No I'm back with the original problem... I don't get it.

[s-martin]

So it didn't work, later it worked and now it doesn't work anymore again?

You should check for service outages on the Spotify website.

Or maybe you could upgrade to latest Iris version as mentioned in #894, but I didn’t see any related changes in the changelog of 3.46:
https://github.com/jaedb/Iris/releases

[drybx]

So it didn't work, later it worked and now it doesn't work anymore again?

Exactly!

You should check for service outages on the Spotify website.

Couldn't find anything unfortunytely.

Or maybe you could upgrade to latest Iris version as mentioned in #894

Already did that, but as Iris is just the frontend I don't think the solution can be found there.

[s-martin]

That sounds really weird.

I would still guess that could be some hiccups on Spotify server side or the internet connection
But I would expect something to see in journalctl -u mopidy.

[drybx]

Unfortunately the issue persists (I was hoping that there would be some kind of improvement on the server side).
journalctl -u mopidy gives me two error messages (and I assume that they are unrelated):
ERROR [MainThread] mopidy.internal.log Loading logging config '/etc/mopidy/logging.conf' failed. 'formatters'
ERROR [SpotifyEventLoop] spotify.session Spotify login error: <ErrorType.UNABLE_TO_CONTACT_SERVER: 8>

PS: My account is a Spotify family account, which is a paid account. I did some research if that kind of account creates any problems but couldn't find anything. It would surprise me though, as is has worked before.

[s-martin]

This issue (although from February)
mopidy/mopidy-spotify#257 mentions that this is probably a issue on Spotify side.

I guess that's not a very satisfying answer for you, but right now I don't have any other ideas.

[s-martin]

#901 reported the same issue, workaround was to restart mopidy.

Could you try sudo systemctl restart mopidy and try again, if this works then?

[Chapuisat]

I have the same issue, see #901

What helps is to restart the mopidy server manually (e.g. out of Phoniebox under Folder&Files, you can click on scan music library) and then mopidy is able to login to spotify without any errors and then you can play Spotify. I have no idea yet why this happens and why mopidy is not able to login directly after booting.

[Chapuisat]

Again, directly after booting:

Apr 15 10:58:30 raspberrypi mopidy[453]: ERROR    [SpotifyEventLoop] spotify.session Spotify login error: <ErrorType.UNABLE_TO_CONTACT_SERVER: 8>
Apr 15 10:58:31 raspberrypi mopidy[453]: ERROR    [SpotifyBackend-6] mopidy_spotify.web OAuth token refresh failed: Unknown error.
Apr 15 10:58:31 raspberrypi mopidy[453]: ERROR    [SpotifyBackend-6] mopidy_spotify.web Failed to load Spotify user profile

and after sudo systemctl restart mopidy

Apr 15 11:00:11 raspberrypi mopidy[1063]: INFO     [SpotifyBackend-6] mopidy_spotify.web Logged into Spotify Web API as kfuzej1pajj038nhge0w1af92
Apr 15 11:00:11 raspberrypi mopidy[1063]: INFO     [SpotifyEventLoop] mopidy_spotify.backend Logged in to Spotify in online mode
Apr 15 11:00:47 raspberrypi mopidy[1063]: INFO     [SpotifyBackend-6] mopidy_spotify.playlists Refreshed 18 Spotify playlists

[Chapuisat]

I know this is a bad, maybe temporary, solution, but could we just write a script to automatically restart the mopidy server as the last step of the booting procedure? How could we do that?

[s-martin]

Most useful could be to add a prerequisite to the mopidy.service that internet connection must be available.

[Chapuisat]

Most useful could be to add a prerequisite to the mopidy.service that internet connection must be available.

How? :-)

[kingosticks]

There are two parts to Mopidy-spotify:

• libspotify - Spotify's closed-source c library, mostly just used for playback these days. Spotify's libspotify servers sometimes have issues. In the logs this is spotify.session and SpotifyEventLoop.
• Web API - a newer interface used for searching, playlists, images. In the logs this is spotify.web.

Each part will login separately (using different Spotify servers and different methods) and it happens in the same order that I specified them above.

Between these logs and the ones in #901, sometimes the session fails to connect but web is OK, and sometimes both fail. But I don't see anywhere where the session connects but web doesn't. To me that sounds like either the Internet connection sometimes becomes ready between the two logins. Or Spotify's libspotify servers have an issue. They actually have a pool of load balancing servers and it's been known for some of them to sometimes not work reliably but this is usually fixed within a few days.

To me, the Internet connection not being ready is far more likely. The delay to connect will vary on your network load, particularly if you are using WiFi. I have no idea how your systems are configured but you should be enabling the raspi-config option that waits for the Internet to become ready during boot. If you could confirm you have enabled that setting we can go from there.

[Chapuisat]

Yes, I'm using WiFi to connect.
How can I enable the option in raspi-config that waits for the Internet to become ready during boot? I didn't enable this proactively? How can I check?

[Chapuisat]

That's it! Solved. Followed this steps for the raspi-config wifi boot option:

https://raspberrypi.stackexchange.com/questions/45769/how-to-wait-for-networking-on-login-after-reboot

Now rebooted twice and all looks good, Spotify plays immediately. No login issues any more.

[kingosticks]

That's great. I'd recommend that be made the default setting for this project, if possible.

[drybx]

First of all, I'm very happy we got to the bottom of this and found at least a workaround.

That's it! Solved. Followed this steps for the raspi-config wifi boot option:
https://raspberrypi.stackexchange.com/questions/45769/how-to-wait-for-networking-on-login-after-reboot

The only concern I have is offline functionalty. In an ideal scenario I would like my phoniebox to work without internet connection as well. AFAIK mopidy is needed even for playing offline files, correct?
Hence the phoniebox wouldnt boot all the way if there is no internet connection if set up this way.

[s-martin]

The only concern I have is offline functionalty. In an ideal scenario I would like my phoniebox to work without internet connection as well.

I agree with you that a default prerequisite for internet connectivity may not work for all users, especially Classic edition without streaming.

AFAIK mopidy is needed even for playing offline files, correct?

Yes, that's right the Spotify edition also uses mopidy for local music.

I think we should at least document this in the Troubleshooting FAQ.
Maybe we should also add a more meaningful button/label to the web UI, so users could restart mopidy.

[kingosticks]

I see, apologies, I didn't realise you also had that use-case. Please ignore my suggestion!

[s-martin]

@MiczFlor, it might be a good first step to add some more details in the web UI that a scan also restarts Mopidy. So users could just click to fix that issue.

[kingosticks]

Before I make another bad suggestion because I don't know much about this project, can I just check that you know Mopidy-Local v3.x can update the local database without requiring a restart? i.e. It will start using the updated database once the local scan completes.

[drybx]

Did anyone come up with a way to avoid the issues discussed in this thread whilst maintaining offline functionality.

I'm thinking of all the hours our little ones have to spend in the car while on their way for summer vacation and all the good a functioning phoniebox could do to their parents :-)

[s-martin]

If you click in the web UI to rescan the library after a reboot, mopidy is also restarted, which should fix the issue (until next reboot).
A little bit of an ugly workaround, but should help.

[drybx]

Thanks!
However, I am not quite sure how that would work in practice in an environment where I don't have internet access (hence I can't access the web UI).

[s-martin]

Yes, you are right.
There's a hotspot mode since 2.0, but this could need some configuration.

What you could try is to use a RFID card and hack a system command triggered by the RFID card.

[domu83]

@s-martin You need to keep in mind that mopidy needs to be at least once connected in order to play local content. If mopidy does not connect the local playout is not possible. see #898

[s-martin]

Hm, I think you're right.

But that would mean that a offline mode is not possible for Spotify edition.

So we could add a fix to wait for internet connection before start of mopidy in Spotify edition.

But for @JuliH29 and others, their use case to use Spotify edition also in offline mode would not work.

[domu83]

I'm bit confused when you say Spotify edition. My understanding was, that there is no classic version anymore. When you say Spotify edition, does that just mean that during the installation the Spotify credentials are entered?
So my understanding is that with Mopidy there is no offline functionality possible. With that I understand that offline is never possible because Mopidy is always used?

A workaround could be using out of the box solutions like Raspotify or Shairport Sync

[drybx]

Thank you for all your comments.
So to summarize we can say, that at the moment the Spotify Edition does not offer offline funtionality (even for local files).

For me that would definitely be a feature which would make Phoniebox even stronger.

[kingosticks]

I'm a little confused about one thing still. If you don't configure the Spotify username or password, Mopidy-Spotify will not start and it won't cause any problem. It also makes no sense to configure the Spotify credentials for an offline system that can't possibly use Spotify. What am I missing? Is setting the Spotify credentials somehow non-optional?

[drybx]

The thing is that AFAIK even though there is only one installer now (in contrast to two separate installers before) which installs eithers the Spotify edition or the classic edition (so two different pieces of software).

In my eyes (and my use case) it makes total sense to configure spotify for an offline system. You might use Phoniebox in an environment where you have internet access (at home) most of the time but would also like to use it in other places where internet access is not guaranteed (in your garden, in your car, train...).

[domu83]

@kingosticks thanks for confirming.
I think it's about the flexibility. So for me/my kids the use case would be at home with spotify streams but also take the box to the grandparents where no wifi access is available to just play the local content or as JuliH29 mentioned, during long distance drives for vacation.

[kingosticks]

Right OK, that explains it, thanks both. So these systems are sometimes offline, and sometimes online. This is not a traditional usecase for a server so I don't think Mopidy behaves unreasonably. However, I'll see if there's something simple we can improve in Mopidy-Spotify to handle this situation better.

[drybx]

So these systems are sometimes offline, and sometimes online.

Exactly.

However, I'll see if there's something simple we can improve in Mopidy-Spotify to handle this situation better.

That would be great!

[s-martin]

The thing is that AFAIK even though there is only one installer now (in contrast to two separate installers before) which installs eithers the Spotify edition or the classic edition (so two different pieces of software).

Yes, that's correct. The install script let's you choose between Classic edition (uses MPD, very fast startup and works offline, too) and Spotify edition (uses mopidy, works only (to be confirmed) online, somehow slower startup, can access Spotify premium accounts).

While I see the use cases working offline with Spotify edition, too, my current understanding is that right now this is not possible. If @kingosticks finds a workaround/solution that would be great. Otherwise I think we have reached a dead end here.

[AnnaHerbstkind]

Hey guys, this is exactly my use case too. Using Spotify at home and local files on the road just like the "Toniebox" works. I was surprised to read that this is not possible at the moment. @kingosticks if you find a workaround or rather solution for this i would be very greatful!

[s-martin]

I added a description to the Troubleshooting FAQ:
https://github.com/MiczFlor/RPi-Jukebox-RFID/wiki/Troubleshooting-FAQ#spotify-wont-play-but-after-scan-of-library-it-works

[s-martin]

Currently no real fix available, workaround described in https://github.com/MiczFlor/RPi-Jukebox-RFID/wiki/Troubleshooting-FAQ#spotify-wont-play-but-after-scan-of-library-it-works

[kingosticks]

We also just merged this, which should help: mopidy/mopidy#1946

[s-martin]

We also just merged this, which should help: mopidy/mopidy#1946

Cool. Do you have an estimate for the release of 3.1?

[kingosticks]

-3 days ago :)
https://github.com/mopidy/mopidy/releases/tag/v3.1.0

[kingosticks]

Although, I'm not actually sure that has been pushed to apt.mopidy.com yet... We do have a little lag there usually.

[drybx]

-3 days ago :)
https://github.com/mopidy/mopidy/releases/tag/v3.1.0

Do I understand correctly that this would improve the performance of an online (=Spotify) installation but wouldn't provide offline functionality to it?

[moxli]

I just ran into the same issue @JuliH29 has described in the initial post.

I am running Mopidy 3.1.0 with Mopidy-Spotify (4.1.0, I manually upgraded from 4.0.1 to check if this fixes the issue).

I have enabled the "Wait for Network on Boot" option in the raspi-config.
I am using WiFi if this matters.

After entering my Spotify (Familiy Premium Account) credentials in the mopidy config file the account was recognized in Iris, I could see my Playlist and the Covers of songs in the Phoniebox UI.
In the Iris settings the account was marked as read-only/restricted access. After logging in through Iris the account shows up as "Autorized" but this did not fix following issue for me.

I am getting the following error directly after booting:

INFO [SpotifyBackend-6] mopidy_spotify.web Logged into Spotify Web API as XXXX ERROR [SpotifyEventLoop] spotify.session Spotify login error: <ErrorType.UNABLE_TO_CONTACT_SERVER: 8>

When trying to play a song through the Phoniebox UI (I see the cover) by adding the Spotify URI to a card I see the following in the mopidy log:

[SpotifyBackend-6] mopidy_spotify.lookup Failed to lookup 'spotify:track:7HXBU3UqdSSzKYMFSLpAKD': Session must be logged in and online to load objects: <ConnectionState.LOGGED_OUT: 0>

In Iris I can search Spotify without any issues. When I try to start a song directly through Iris I get the same error:

[SpotifyBackend-6] mopidy_spotify.lookup Failed to lookup 'spotify:track:10waMsh44ojvTwXAIy213n': Session must be logged in and online to load objects: <ConnectionState.LOGGED_OUT: 0>

Manually restarting mopidy, a complete reboot or clicking the "scan library files" button in the Phoniebox UI does not fix the issue for me.

I am experiencing this on a fresh install since at least three days.
Do you have any idea what the issue could be?

From what I have read on multiple sources the issue should not be regarding the network connection because it's mopidy 3.1.0, I tried manually restarting mopidy and the "wait for network on boot" option is enabled.