Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata Providers #5

Open
7 of 17 tasks
kellnerd opened this issue May 16, 2024 · 19 comments
Open
7 of 17 tasks

Metadata Providers #5

kellnerd opened this issue May 16, 2024 · 19 comments
Labels
provider Metadata provider

Comments

@kellnerd
Copy link
Owner

kellnerd commented May 16, 2024

List of sources/websites for which a metadata provider has been implemented or requested.

Leave a comment which includes at least a link to an example release for a quick provider request, or create a separate issue which is named after the provider and labeled with provider Metadata provider if you have more research to share.
Detailed requests for sources with an open API and good documentation are more likely to be implemented.
Edit: Let's be honest, every request which contains more details than just the name or URL of the source is probably worth its own issue already.

If you plan to work on a provider, be it doing more research or actually implementing it, please create a separate issue which is named after the provider and ask for it being assigned to you.

Open JSON API

Restricted JSON API

Tokens and other secrets should not be included in this repository but loaded from environment variables.

HTML Scraping

Uncategorized Requests

@kellnerd kellnerd pinned this issue May 16, 2024
@kellnerd kellnerd added the provider Metadata provider label May 29, 2024
@tigerman325
Copy link

My free time has been scarce these past few months. I like it, but I'll likely hold off on using it often until Spotify is included. It's puzzling why Spotify now limits visibility to only see releases in one's own country, but thankfully, a-tisket remains reliable for those releases if you know the URL. With some luck, I'll have the opportunity to explore it further.

@wileyfoxyx
Copy link

wileyfoxyx commented Jun 4, 2024

Would be interesting to see some kind of Yandex Music implantation is such service.

Example of an album: https://music.yandex.ru/album/12353342
Open JSON API: https://api.music.yandex.net/albums/12353342/ (or https://api.music.yandex.net/albums/12353342/with-tracks for additional info on tracks from album, such as the distributor of release). You might need VPN to open those, but here's a mirror for the "with-tracks" response: https://www.jsonkeeper.com/b/YKSE

The API does not support neither GTAN nor ISRC. Also the "label" section of response takes the info from the P-line of release and in most cases would remove words "Productions", "Music", "Publishing" and etc., as well as split one label onto multiple ones if there's a slash in its name (like here).

They also support showing whether it's an album, single, podcast or an audiobook (since they all have a link of https://music.yandex.ru/album/album_id).

There's also an unofficial implantation of an API at https://github.com/MarshalX/yandex-music-api/releases

@UltimateRiff
Copy link

one big provider (or two, depending on how you count it) I see missing is YouTube/YouTube Music. I feel like it might be right to kinda treat them seperately (as a user, not a developer)

the main things I know to look out for on YouTube

  • metadata in the description is not at all standardized, perhaps save for distributed content (i.e. from distributors like DistroKid)
  • a single video can have different titles, descriptions, and possibly different audio tracks per region
  • often an album will be released as a single video with chapter markers for each track
  • video titles are not standardized, sometimes containing the artist name or [official video] or other nonsense, which maybe should or shouldn't be removed from MusicBrainz submissions

I can make a ticket or two for this pair if you'd like~

@wileyfoxyx
Copy link

@UltimateRiff, I'd either would and would not count YouTube and YouTube Music as separate, mostly because of the so-called Art Tracks (that look like ordinary track in any music service on YTM yet as an ordinary video with "Provided to YouTube by..." line in the description on YT)

@reubot
Copy link

reubot commented Jun 5, 2024

7digital.com has API docs at https://docs.7digital.com .

@c0b41
Copy link

c0b41 commented Jun 5, 2024

YouTube.js support youtube music api, no token required and GTIN search works.
Also support deno.

@kellnerd
Copy link
Owner Author

kellnerd commented Jun 5, 2024

@wileyfoxyx Thank you for the detailed request, this is enough to warrant a separate ticket. Can you please make one? (I will then leave my comments over there.)

@UltimateRiff One YouTube ticket should be enough for the start, we can always create a new ticket if it turns out there will be multiple providers and only one of them is implemented initially.
And the library suggested by @c0b41 should also be linked from there. Although I am not sure whether we will just have our own simple implementation inspired by the library, since we need to make requests in a way which is compatible with Harmony's caching.

If I am being honest, even a request with just a link to a good API documentation is enough for a separate issue. Can you please make one @reubot? I will clarify that in the description of this issue.

@phw
Copy link
Collaborator

phw commented Jun 5, 2024

PR #11 adds initial support for Tidal

@phw phw mentioned this issue Jun 7, 2024
This was referenced Jun 11, 2024
@quantenzitrone
Copy link

I was about to suggest Amazon as a metadata provider, but they don't support GTINs for albums.

So right now, Amazon could only work as a standalone provider like Bandcamp.
They have a restricted JSON API that is in Beta that seems quite good, but for authentication, you need to contact the Amazon Business Development representative to get access to the API beta.

@quantenzitrone
Copy link

quantenzitrone commented Aug 13, 2024

boomplay has an api that includes getting albums by UPC and songs by ISRC.

However, you need to register an application and get an access token, and there doesn't seem to be any documentation on how to register an application.

https://developer.boomplay.com/openApi/track/GetTrackByIsrc.html

Edit: I contacted their customer support, if it was possible to get access to their API, but got no answer :(.

@quantenzitrone
Copy link

im just looking up random music streaming platforms at this point...

KKBOX is a Music Streaming Platform from Taiwan that is only available in Hong Kong, Japan, Macau, Malaysia, Singapore and Taiwan.
They have a restricted JSON api, that, similar to spotify, requires registering an application, which is free (as in gratis) and fairly easy.
While all the registering has to be done from an IP Address in one of the above countries (I managed to do that through a VPN) the API itself has no geo restrictions.

They also have good documentation https://docs-en.kkbox.codes/docs, however, they don't seem to have GTINs in their metadata.

You can use my access token for testing until it expires: _oMkgQA5yo5GiY5IUvKJ6Q==.
The access tokens seem to be valid for one month.

@quantenzitrone
Copy link

QQ Music has an api that doesn't include GTINs and ISRCs.
The Documentation is completely in Chinese and getting an access token requires creating an account and asking them for authentication. So probably not worth it.

@HibiscusKazeneko
Copy link

HibiscusKazeneko commented Sep 6, 2024

Traxsource doesn't appear to have an API, so HTML scraping is probably the best we can do to get data.
Example release: https://www.traxsource.com/title/2292000/
Example label link: https://www.traxsource.com/label/69750/

@quantenzitrone
Copy link

quantenzitrone commented Sep 11, 2024

https://www.junodownload.com has or at least had a restricted API.
The docs are on archive.org: https://web.archive.org/web/20150501040915/http://www.junodownload.com:80/api/docs/JunoAPI_SEARCH_v1.3.1.pdf
It uses xml, and I cannot find any references to GTIN UPC Barcode or ISRC in the document, so it's likely not worth it to invest time into implementing their API.

@Xythium
Copy link

Xythium commented Oct 30, 2024

for junodownload, api endpoints that i have found that still work (and dont require an api key which might be impossible to get):

i think junodownload support can be added but it might require being able to lookup with multiple links, so you could have juno & bandcamp links for example (sites without barcode lookup) in addition to a spotify link for example so it is able to search other apis

@RyanThaDude
Copy link

It seems that Qobuz can now search via GTIN/barcode now: catalog/search?query=4099964127652

Of course the API is still restricted :-/

@tigerman325
Copy link

It seems that Qobuz can now search via GTIN/barcode now: catalog/search?query=4099964127652

Of course the API is still restricted :-/

I guess I'm doing something wrong, because it doesn't work for me.

@RyanThaDude
Copy link

It seems that Qobuz can now search via GTIN/barcode now: catalog/search?query=4099964127652
Of course the API is still restricted :-/

I guess I'm doing something wrong, because it doesn't work for me.

What I'm finding so far is that it has to precisely match the barcode, and I'm pretty sure Qobuz uses EAN (13 digit length) for their barcodes. Searching via barcode also works in the web player, but NOT for downloads.

@quantenzitrone
Copy link

quantenzitrone commented Nov 24, 2024

try

curl \
  --header 'X-User-Auth-Token: <your token>' \
  --url "https://www.qobuz.com/api.json/0.2/catalog/search?query=4099964127652&app_id=579939560"

don't tell anyone, but there are tokens here

the result should be:
{
  "query": "4099964127652",
  "albums": {
    "limit": 50,
    "offset": 0,
    "total": 1,
    "items": [
      {
        "maximum_bit_depth": 24,
        "image": {
          "small": "https://static.qobuz.com/images/covers/qa/wz/bcspi7bw4wzqa_230.jpg",
          "thumbnail": "https://static.qobuz.com/images/covers/qa/wz/bcspi7bw4wzqa_50.jpg",
          "large": "https://static.qobuz.com/images/covers/qa/wz/bcspi7bw4wzqa_600.jpg",
          "back": null
        },
        "media_count": 1,
        "artist": {
          "image": null,
          "name": "Memphis May Fire",
          "id": 570185,
          "albums_count": 44,
          "slug": "memphis-may-fire",
          "picture": null
        },
        "artists": [
          {
            "id": 570185,
            "name": "Memphis May Fire",
            "roles": [
              "main-artist"
            ]
          }
        ],
        "upc": "4099964127652",
        "released_at": 1728424800,
        "label": {
          "name": "Rise Records",
          "id": 7409,
          "albums_count": 1212,
          "supplier_id": 5,
          "slug": "rise-records-1"
        },
        "title": "Hell Is Empty",
        "qobuz_id": 293659650,
        "version": null,
        "url": "https://www.qobuz.com/fr-fr/album/hell-is-empty-memphis-may-fire/bcspi7bw4wzqa",
        "duration": 531,
        "parental_warning": false,
        "popularity": 0,
        "tracks_count": 3,
        "genre": {
          "path": [
            116,
            115
          ],
          "color": "#5eabc1",
          "name": "Hard Rock",
          "id": 115,
          "slug": "hard-rock"
        },
        "maximum_channel_count": 2,
        "id": "bcspi7bw4wzqa",
        "maximum_sampling_rate": 48,
        "articles": [],
        "release_date_original": "2024-10-09",
        "release_date_download": "2024-10-09",
        "release_date_stream": "2024-10-09",
        "purchasable": true,
        "streamable": false,
        "previewable": true,
        "sampleable": true,
        "downloadable": true,
        "displayable": true,
        "purchasable_at": 1728424800,
        "streamable_at": 1728424800,
        "hires": true,
        "hires_streamable": false
      }
    ]
  },
  "tracks": {
    "limit": 50,
    "offset": 0,
    "total": 0,
    "items": []
  },
  "artists": {
    "limit": 50,
    "offset": 0,
    "total": 0,
    "items": []
  },
  "playlists": {
    "limit": 50,
    "offset": 0,
    "total": 0,
    "items": []
  },
  "stories": {
    "limit": 50,
    "offset": 0,
    "total": 0,
    "items": []
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
provider Metadata provider
Projects
None yet
Development

No branches or pull requests