ValidationError when requesting a playlist containing an episode

[Kuwertzel]

Issue

When requesting a playlist containing a show (podcast) episode, pydantic throws 10 ValidationErrors indicating that required fields are missing or containing the wrong values.
When I request the same playlist using the spotify developer portal directly, all fields seem to be present.

Looking at the traceback below, the error pydantic_core._pydantic_core.ValidationError: 10 validation errors for FullPlaylistTrack indicates that a FullPlaylistTrack instead of a FullPlaylistEpisode was recognized, which is most likely why the data can't be validated properly.

Traceback (most recent call last):
  File "C:\Users\user\source\repos\spotify-enhanced\mre.py", line 9, in <module>
    playlist_containing_podcasts = spotify.playlist('59coK2uTZ7QXL47iNFeuqD')
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\source\repos\tekore\src\tekore\_client\api\playlist\view.py", line 43, in wrapper
    return post_func(json)
           ^^^^^^^^^^^^^^^
  File "C:\Users\user\source\repos\tekore\src\tekore\_client\process.py", line 29, in post_func
    return type_(**json) if json is not None else None
           ^^^^^^^^^^^^^
  File "C:\Users\user\source\repos\tekore\src\tekore\_model\serialise.py", line 48, in __init__
    super().__init__(**data)
  File "C:\Users\user\AppData\Roaming\Python\Python311\site-packages\pydantic\main.py", line 165, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
  File "C:\Users\user\source\repos\tekore\src\tekore\_model\serialise.py", line 48, in __init__
    super().__init__(**data)
  File "C:\Users\user\AppData\Roaming\Python\Python311\site-packages\pydantic\main.py", line 165, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
  File "C:\Users\user\source\repos\tekore\src\tekore\_model\serialise.py", line 48, in __init__
    super().__init__(**data)
  File "C:\Users\user\AppData\Roaming\Python\Python311\site-packages\pydantic\main.py", line 165, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
  File "C:\Users\user\source\repos\tekore\src\tekore\_model\serialise.py", line 48, in __init__
    super().__init__(**data)
  File "C:\Users\user\AppData\Roaming\Python\Python311\site-packages\pydantic\main.py", line 165, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 10 validation errors for FullPlaylistTrack
artists
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
disc_number
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
is_local
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
preview_url
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
track_number
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
album
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
external_ids
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
popularity
  Field required [type=missing, input_value={'explicit': False, 'audi...7g3E5DAfENjfEwpVP2Rr5e'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/missing
episode
  Input should be False [type=literal_error, input_value=True, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.3/v/literal_error
track
  Input should be True [type=literal_error, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.3/v/literal_error

Process finished with exit code 1

Unfortunately I can't seem to find where/how you determine what type of track should be validated, so I couldn't create a PR to fix it directly.

Expected behavior

spotify.playlist() should try to validate episodes in the tracks field as FullPlaylistEpisode instead of FullPlaylistTrack

Steps to reproduce

Use spotify.playlist() to request a playlist containing at least track one of type episode. I used this playlist for testing.

Minimal Working Example:

import tekore as tk

# authentication
conf = tk.config_from_file('userdata/tekore.cfg', return_refresh=True)
user_token = tk.refresh_user_token(*conf[:2], conf[3])
spotify = tk.Spotify(user_token)

# request any playlist containing at least one podcast
playlist_containing_podcasts = spotify.playlist('59coK2uTZ7QXL47iNFeuqD')

[felix-hilden]

Hi and thank you for submitting the issue! We've made some related changes recently, so please let me know if upgrading Tekore helps. But if it doesn't I'll see what this is about! And to be fair, I saw something similar in our daily test run, so it might be a recent change. The validation error is misleading because the type is a union (of track and episode) so Pydantic just spits out everything even though it might be really close to one of them.

[Kuwertzel]

Hi, I discovered the bug on the current version 5.2.1.
I also just tested on v5.2.0, v5.1.0 and v5.0.0. All of them run into the same error.

Thank you for your help! This is such an awesome library 👏

[felix-hilden]

Thank you 🙏 one thing would help a lot still: could you upgrade Pydantic too? In our tests I see that Show publisher has an object inside rather than the string advertised in the docs and the OpenAPI spec, and is_playable is missing despite being "required". Our error messages are a bit different but I'm curious if you could be seeing something similar.

[Kuwertzel]

Yep, my pydantic version was quite outdated. I upgraded to pydantic 2.5.2 and pydantic-core 2.14.5.

I get a more verbose traceback now. The is_playable is marked as missing and the FullPlaylistEpisode.show.publisher indeed appears to be an object, judging from the traceback.

tracks.items.0.track.FullPlaylistEpisode.is_playable
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]

tracks.items.0.track.FullPlaylistEpisode.show.publisher
  Input should be a valid string [type=string_type, input_value={'name': 'detektor.fm – Das Podcast-Radio'}, input_type=dict]

To be fair though, pydantic still complains about seemingly every field.

In case it helps; the full traceback is appended, with only errors for the available_markets items abbreviated:

Traceback (most recent call last):
  File "C:\Users\user\source\repos\spotify-enhanced\mre.py", line 9, in <module>
    playlist_containing_podcasts = spotify.playlist('6qODgO8aCg7y35VKI9wfBI')  # Test playlist
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\source\repos\spotify-enhanced\venv\Lib\site-packages\tekore\_client\api\playlist\view.py", line 43, in wrapper
    return post_func(json)
           ^^^^^^^^^^^^^^^
  File "C:\Users\user\source\repos\spotify-enhanced\venv\Lib\site-packages\tekore\_client\process.py", line 29, in post_func
    return type_(**json) if json is not None else None
           ^^^^^^^^^^^^^
  File "C:\Users\user\source\repos\spotify-enhanced\venv\Lib\site-packages\tekore\_model\serialise.py", line 48, in __init__
    super().__init__(**data)
  File "C:\Users\user\source\repos\spotify-enhanced\venv\Lib\site-packages\pydantic\main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 193 validation errors for FullPlaylist
tracks.items.0.track.FullPlaylistTrack.artists
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.disc_number
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.is_local
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.preview_url
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.track_number
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.album
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.external_ids
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.popularity
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistTrack.episode
  Input should be False [type=literal_error, input_value=True, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.5/v/literal_error
tracks.items.0.track.FullPlaylistTrack.track
  Input should be True [type=literal_error, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.5/v/literal_error
tracks.items.0.track.FullPlaylistEpisode.is_playable
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.FullPlaylistEpisode.show.publisher
  Input should be a valid string [type=string_type, input_value={'name': 'detektor.fm – Das Podcast-Radio'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/string_type
tracks.items.0.track.LocalPlaylistTrack.id
  Input should be None [type=none_required, input_value='6zxFogoXXfgP1hevPFo8hf', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/none_required
tracks.items.0.track.LocalPlaylistTrack.href
  Input should be None [type=none_required, input_value='https://api.spotify.com/.../6zxFogoXXfgP1hevPFo8hf', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/none_required
tracks.items.0.track.LocalPlaylistTrack.album
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.artists
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.available_markets.0
  Input should be None [type=none_required, input_value='AD', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/none_required

[...]

tracks.items.0.track.LocalPlaylistTrack.available_markets.168
  Input should be None [type=none_required, input_value='ZW', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/none_required
tracks.items.0.track.LocalPlaylistTrack.disc_number
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.external_ids
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.is_local
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.popularity
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.preview_url
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.track_number
  Field required [type=missing, input_value={'explicit': False, 'audi...6zxFogoXXfgP1hevPFo8hf'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
tracks.items.0.track.LocalPlaylistTrack.episode
  Input should be False [type=literal_error, input_value=True, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.5/v/literal_error
tracks.items.0.track.LocalPlaylistTrack.track
  Input should be True [type=literal_error, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.5/v/literal_error

[felix-hilden]

Thank you! The episode model has also been updated to have restrictions and to deprecate language, so I'll update them at the same time.

[felix-hilden]

And apparently markets in the playlist episode version :P and also local track duration_ms is an object. Maybe they're having a bad week over there 😅

[felix-hilden]

I cannot successfully run the test suite locally either due to unrelated errors. So I'll just release this along with other minor stuff. Merry Christmas!

[Kuwertzel]

Sounds good to me! I'll probably work with the library in the next few days, so I'll see if anything is broken :D
Thank you and Happy Holidays!