Fetch song metadata by videoId #42

JGMEYER · 2020-07-09T12:33:18Z

[Creating ticket per CONTRIBUTING.rst]

Context:
YouTube Music urls already provide the id to the video. It would be nice to be able to fetch a video's metdata, e.g. name, artists from this videoId. Similar to get_album() for albums. get_song(self, videoId) could return a song's metadata.

How this feature is missing:
From reading the documentation, there doesn't seem to be an endpoint to retrieve this information or a set of chained function calls to achieve this use case.

Acceptance criteria:
This ticket is to gather information on the set of requirements and to implement the method in class YTMusic, if currently feasible.

This ticket can be closed under any of the following conditions:

creating such an endpoint is infeasible, given limitiations
this feature is explicitly chosen to be unsupported by this api
get_song(self, videoId) has been implemented in YTMusic (with corresponding unit/integration tests) for fetching song metadata

I'm not yet sure about the limitations on the api for fetching this information, or futhermore if this would somehow be blocked by #10. I (@JGMEYER) am open to help develop this feature, but may need some guidance.

The text was updated successfully, but these errors were encountered:

sigma67 · 2020-07-09T14:26:08Z

This should be possible. YouTube Music uses a different API (www.youtube.com/get_video_info) than the one usually used in ytmusicapi for this purpose (music.youtube.com). But I believe the request should work using the same cookies that the rest of ytmusicapi uses. The endpoint was reverse engineered here: https://tyrrrz.me/blog/reverse-engineering-youtube

The response needs to be URL decoded, afterwards you can find the interesting parts in player_response (JSON containing all the info), more specifically the key videoDetails. Below I've extracted an example. Is this what you had in mind?

The individual items probably need to be processed a bit, since they still contain '+' instead of spaces.

"videoDetails": {
    "videoId": "e_S9VvJM1PI",
    "title": "Icon+For+Hire+-+Make+A+Move",
    "lengthSeconds": "184",
    "keywords": [
      "Icon",
      "For",
      "Hire",
      "Make",
      "Move",
      "Tooth",
      "Nail",
      "(TNN)",
      "Rock"
    ],
    "channelId": "UCKvT-8xU_BTJGvsQ5lR23TQ",
    "isOwnerViewing": false,
    "shortDescription": "Music+video+by+Icon+For+Hire+performing+Make+A+Move.+(P)+(C)+2011+Tooth+&+Nail+Records.+All+rights+reserved.+Unauthorized+reproduction+is+a+violation+of+applicable+laws.++Manufactured+by+Tooth+&+Nail,\n\n#IconForHire+#MakeAMove+#Vevo+#Rock+#VevoOfficial+#OfficialMusicVideo",
    "isCrawlable": true,
    "thumbnail": {
      "thumbnails": [
        {
          "url": "https://i.ytimg.com/vi/e_S9VvJM1PI/hqdefault.jpg?sqp=-oaymwEYCKgBEF5IVfKriqkDCwgBFQAAiEIYAXAB&rs=AOn4CLDk8hJWlOA-E5hu_k1jfI_Lcb-uRQ",
          "width": 168,
          "height": 94
        },
        {
          "url": "https://i.ytimg.com/vi/e_S9VvJM1PI/hqdefault.jpg?sqp=-oaymwEYCMQBEG5IVfKriqkDCwgBFQAAiEIYAXAB&rs=AOn4CLCQ-jftDxm4Utz5FvbK_LM1qwBQpA",
          "width": 196,
          "height": 110
        },
        {
          "url": "https://i.ytimg.com/vi/e_S9VvJM1PI/hqdefault.jpg?sqp=-oaymwEZCPYBEIoBSFXyq4qpAwsIARUAAIhCGAFwAQ==&rs=AOn4CLAyuGBcLx6EBJhU3K81PR57EMWFBQ",
          "width": 246,
          "height": 138
        },
        {
          "url": "https://i.ytimg.com/vi/e_S9VvJM1PI/hqdefault.jpg?sqp=-oaymwEZCNACELwBSFXyq4qpAwsIARUAAIhCGAFwAQ==&rs=AOn4CLCKxtoRcbOOzMSjI0Qrns1HPOZqGA",
          "width": 336,
          "height": 188
        },
        {
          "url": "https://i.ytimg.com/vi/e_S9VvJM1PI/maxresdefault.jpg",
          "width": 1920,
          "height": 1080
        }
      ]
    },
    "averageRating": 4.8960409,
    "allowRatings": true,
    "viewCount": "43342796",
    "author": "IconForHireVEVO",
    "isPrivate": false,
    "isUnpluggedCorpus": false,
    "isLiveContent": false
  },

JGMEYER · 2020-07-09T14:31:05Z

That’s pretty close. The disadvantage I see here is that you have to parse the video ‘title’ to “best guess” the song ‘name’ and ‘artist’. Versus being able to retrieve these values explicitly in a response from the json. Maybe the namings of YTMusic videos are better standardized than YouTube’s? Or maybe they’re one in the same. I’m still familiarizing myself with the platform.

sigma67 · 2020-07-09T14:53:15Z

I think you actually get that info if it's a song (and not a video as above). I tried with a public track from YTMusic:

"videoDetails": {
    "videoId": "drknbHJ1MOs",
    "title": "Horyzon",
    "lengthSeconds": "199",
    "keywords": [
      "ARTY",
      "Horyzon"
    ],
    "channelId": "UCRQ_TyUQ_gfGxpEMpTNgCnQ",
    "isOwnerViewing": false,
    "shortDescription": "Provided+to+YouTube+by+Armada\n\nHoryzon+·+ARTY\n\nHoryzon\n\n℗+Armada+Music+B.V.\n\nReleased+on:+2020-07-03\n\nProducer:+ARTY\nComposer:+Artem+Stolyarov\nMusic++Publisher:+Copyright+Control\nComposer:+Pavel+Esenin\nMusic++Publisher:+Telma+Music+(adm.+by+Kobalt)\n\nAuto-generated+by+YouTube.",
    "isCrawlable": true,
    "thumbnail": {
      "thumbnails": [
        {
          "url": "https://lh3.googleusercontent.com/EXZYuJprL0OUqopZNmgZPS-LeBXaiXBHglfgUSpFPt69KppEucgwPlOdXPpe0f8InImfa9DjGSZcIr0NCw=w60-h60-l90-rj",
          "width": 60,
          "height": 60
        },
        {
          "url": "https://lh3.googleusercontent.com/EXZYuJprL0OUqopZNmgZPS-LeBXaiXBHglfgUSpFPt69KppEucgwPlOdXPpe0f8InImfa9DjGSZcIr0NCw=w120-h120-l90-rj",
          "width": 120,
          "height": 120
        },
        {
          "url": "https://lh3.googleusercontent.com/EXZYuJprL0OUqopZNmgZPS-LeBXaiXBHglfgUSpFPt69KppEucgwPlOdXPpe0f8InImfa9DjGSZcIr0NCw=w226-h226-l90-rj",
          "width": 226,
          "height": 226
        },
        {
          "url": "https://lh3.googleusercontent.com/EXZYuJprL0OUqopZNmgZPS-LeBXaiXBHglfgUSpFPt69KppEucgwPlOdXPpe0f8InImfa9DjGSZcIr0NCw=w544-h544-l90-rj",
          "width": 544,
          "height": 544
        }
      ]
    },
    "averageRating": 5.0,
    "allowRatings": true,
    "viewCount": "844",
    "author": "Arty+-+Topic",
    "isPrivate": false,
    "isUnpluggedCorpus": false,
    "musicVideoType": "MUSIC_VIDEO_TYPE_ATV",
    "isLiveContent": false
  },

and a private track

"videoDetails": {
    "videoId": "gzrmnueiQk8",
    "title": "Drinking+From+the+Bottle",
    "lengthSeconds": "240",
    "channelId": "UCvCk2zFqkCYzpnSgWfx0qOg",
    "isOwnerViewing": false,
    "shortDescription": "Uploaded+to+YouTube+via+YouTube+Music\n\nDrinking+From+the+Bottle",
    "isCrawlable": false,
    "thumbnail": {
      "thumbnails": [
        {
          "url": "https://www.gstatic.com/youtube/media/ytm/images/cover_track_default@1200.png?sqp=CPXnx-oF-oaymwEGCDwQPFgB&rs=ALLJMcKG4HoG-nDcHAjjtvKnA1-TsHbplA",
          "width": 60,
          "height": 60
        },
        {
          "url": "https://www.gstatic.com/youtube/media/ytm/images/cover_track_default@1200.png?sqp=CMnnx-oF-oaymwEGCHgQeFgB&rs=ALLJMcJMYVX-njU3_fFWsT7BvREyrfEBFQ",
          "width": 120,
          "height": 120
        },
        {
          "url": "https://www.gstatic.com/youtube/media/ytm/images/cover_track_default@1200.png?sqp=CJPnx-oF-oaymwEICIACEIACWAE&rs=ALLJMcK6iOZG3iTkwYz1rwS4FUFrFRCYdQ",
          "width": 256,
          "height": 256
        },
        {
          "url": "https://www.gstatic.com/youtube/media/ytm/images/cover_track_default@1200.png?sqp=CL7mx-oF-oaymwEICKAEEKAEWAE&rs=ALLJMcK3PWlHNF_6heb1uzGdlfLe6y5g0g",
          "width": 544,
          "height": 544
        }
      ]
    },
    "averageRating": 0.0,
    "allowRatings": true,
    "viewCount": "0",
    "author": "Music+Library+Uploads",
    "isPrivate": true,
    "isUnpluggedCorpus": false,
    "musicVideoType": "MUSIC_VIDEO_TYPE_PRIVATELY_OWNED_TRACK",
    "isLiveContent": false
  },

The streamingData key contains some interesting info as well, such as technical video details and streaming URLs for different playback qualities.

Here's the full response: get_video_info_json.txt

sigma67 · 2020-07-13T16:41:05Z

I did some more research and it seems YTMusic actually uses the next() endpoint (https://music.youtube.com/youtubei/v1/next) to get the info shown in the player. However, the info provided by that endpoint is even less structured as get_video_info, as it simply provides the "bylines" shown at the bottom, without specifying whether the info is artist, like, views, album etc.

Here's an example for a music video:

"itemSectionRenderer": {
                "contents": [
                  {
                    "musicWatchMetadataRenderer": {
                      "title": {
                        "runs": [
                          {
                            "text": "Born in the U.S.A."
                          }
                        ]
                      },
                      "byline": {
                        "runs": [
                          {
                            "text": "Bruce Springsteen",
                            "navigationEndpoint": {
                              "clickTrackingParams": "CAcQkGsYACITCL2YgOzSyuoCFYXaVQod3twK_Q==",
                              "browseEndpoint": {
                                "browseId": "UCyFqJ_5TyAeTD7rHpLikbKQ",
                                "browseEndpointContextSupportedConfigs": {
                                  "browseEndpointContextMusicConfig": {
                                    "pageType": "MUSIC_PAGE_TYPE_ARTIST"
                                  }
                                }
                              }
                            }
                          }
                        ]
                      },
                      "secondaryByline": {
                        "runs": [
                          {
                            "text": "47 million views"
                          },
                          {
                            "text": " • "
                          },
                          {
                            "text": "211.419  likes"
                          }
                        ]
                      },
                      "secondaryTitle": {
                        "runs": [
                          {
                            "text": "Bruce Springsteen - Born in the U.S.A."
                          }
                        ]
                      },
                      "viewCountText": {
                        "runs": [
                          {
                            "text": "47.242.508 views"
                          }
                        ]
                      }

and another for a regular song:

      "playlist": {
        "playlistPanelRenderer": {
          "title": "Don't Wanna Fall",
          "contents": [
            {
              "playlistPanelVideoRenderer": {
                "title": {
                  "runs": [
                    {
                      "text": "Don't Wanna Fall"
                    }
                  ]
                },
                "longBylineText": {
                  "runs": [
                    {
                      "text": "Seven Lions, Last Heroes & HALIENE • 3:52"
                    }
                  ]
                },

I do believe that the next endpoint should be implemented eventually, as it provides a way to retrieve radio mix playlists for a given song. I believe that YouTube's get_video_info is better suited for getting metadata however. It also provides the YouTube video's auto generated description, which can be parsed in a more structured fashion than YTMusic's unstructured bylines.

JGMEYER · 2020-07-13T17:41:11Z

Good find! Yeah, it does seem the youtube api would be better suited for this with how unstructured the youtube nusic api response is. Probably makes this feature out of this project's scope? I haven't actually gotten the chance yet to follow the reverse engineered steps, but I skimmed it over and it looked straightforward enough. Just need to get some spare time again to tinker with it.

sigma67 · 2020-07-14T08:10:07Z

I already made some progress on the get_video_info implementation and pushed to the branch feature/get_song (referenced above).

Let me know what you think. Feel free to add or change things if necessary

sigma67 · 2020-07-20T08:35:09Z

Are you still interested in this issue @JGMEYER ? Otherwise I'll go ahead and merge it into master

JGMEYER · 2020-07-20T10:30:09Z

Please feel free to merge. I managed to use one of the youtube endpoints for my project from the blog you posted, but I’d be happy to test out any release that contains these changes :)

On Mon, Jul 20, 2020 at 04:35 sigmatics ***@***.***> wrote: Are you still interested in this issue @JGMEYER <https://github.com/JGMEYER> ? Otherwise I'll go ahead and merge it into master — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEKYXLV4HX4BQ5JSO46AZLR4P6VBANCNFSM4OVRZQDQ> .

-- *Justian Meyer* *Georgia Institute of Technology* Computer Science '14

sigma67 added the enhancement New feature or request label Jul 9, 2020

sigma67 pushed a commit that referenced this issue Jul 14, 2020

Feature: add get_song (#42)

d74e532

sigma67 closed this as completed in a579564 Jul 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch song metadata by videoId #42

Fetch song metadata by videoId #42

JGMEYER commented Jul 9, 2020 •

edited

Loading

sigma67 commented Jul 9, 2020 •

edited

Loading

JGMEYER commented Jul 9, 2020

sigma67 commented Jul 9, 2020

sigma67 commented Jul 13, 2020 •

edited

Loading

JGMEYER commented Jul 13, 2020

sigma67 commented Jul 14, 2020

sigma67 commented Jul 20, 2020

JGMEYER commented Jul 20, 2020 via email

Fetch song metadata by videoId #42

Fetch song metadata by videoId #42

Comments

JGMEYER commented Jul 9, 2020 • edited Loading

sigma67 commented Jul 9, 2020 • edited Loading

JGMEYER commented Jul 9, 2020

sigma67 commented Jul 9, 2020

sigma67 commented Jul 13, 2020 • edited Loading

JGMEYER commented Jul 13, 2020

sigma67 commented Jul 14, 2020

sigma67 commented Jul 20, 2020

JGMEYER commented Jul 20, 2020 via email

JGMEYER commented Jul 9, 2020 •

edited

Loading

sigma67 commented Jul 9, 2020 •

edited

Loading

sigma67 commented Jul 13, 2020 •

edited

Loading