Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discogs again #689

Open
PatPanacea opened this issue Mar 16, 2021 · 9 comments
Open

Discogs again #689

PatPanacea opened this issue Mar 16, 2021 · 9 comments
Labels
impossible? Websites that might be impossible to support topic:rules

Comments

@PatPanacea
Copy link

Sample: https://img.discogs.com/ndX3ORCM48WofqvPv9Qh3Ynsdp0=/fit-in/600x595/filters:strip_icc():format(jpeg):mode_rgb():quality(90)/discogs-images/R-15025166-1613956799-5152.png.jpg

@qsniyg
Copy link
Owner

qsniyg commented Mar 16, 2021

Well, they've now removed the method, old urls don't work anymore either (all 404). Now we're back to square one... I'll do some research later, and if you find anything at all, please let me know!

(Edit: I'm adding the "impossible?" tag pre-emptively until a solution can be found. It's very possible they haven't exposed any endpoint to retrieve the original images)

@PatPanacea
Copy link
Author

Thanks so much for your attention and your quick response. I tried a number of URL combinations and got nothing.

@qsniyg qsniyg added the impossible? Websites that might be impossible to support label Mar 17, 2021
qsniyg added a commit that referenced this issue Mar 17, 2021
Not great but better than nothing for now
qsniyg added a commit that referenced this issue Mar 17, 2021
It just gets 600x* though
@qsniyg
Copy link
Owner

qsniyg commented Mar 17, 2021

I've added a few new ways of getting larger images, but it still doesn't get the source, it will just get the "full" (600x) version from a thumbnail. Better than nothing I guess, but still not great.

qsniyg added a commit that referenced this issue Mar 17, 2021
@mjb2010
Copy link

mjb2010 commented Mar 19, 2021

Relevant discussion on Discogs: https://www.discogs.com/forum/thread/828335#8567194

FWIW, it was mentioned several years prior that the libraries they use are called Pillow and Thumbor.

@redactedscribe
Copy link

redactedscribe commented Dec 22, 2021

I've added a few new ways of getting larger images, but it still doesn't get the source, it will just get the "full" (600x) version from a thumbnail. Better than nothing I guess, but still not great.

@qsniyg When viewing a Discogs image in a new tab, the page quickly refreshes and upgrades the URL to the "full" version, but supposedly better image URL is worse than the original. Sometimes the upgraded URL is slightly smaller, say 599 instead of 600px (but this may be expected if 599px is the original width and Discogs is stretching the image to 600?).

I think the lower quality is because the new URL uses :format(webp): in some cases instead of :format(jpeg): (which has more accurate/saturated colours, and a larger filesize). Pressing Back shows the better, jpeg version in this case.

Quality example jpeg->webp example (main cover image: reds clearer and greens sharper in jpeg)
Smaller dimensions example (artist image: upgraded is 599px)

Firefox 95
IMU 0.19.5

@ROpdebee
Copy link

ROpdebee commented Jan 26, 2022

FWIW Discogs seems to have changed the URL format again, now it's e.g. https://i.discogs.com/aRe2RbRXu0g4PvRjrPgQKb_YmFWO3Y0CYc098S8Q1go/rs:fit/g:sm/q:90/h:600/w:576/czM6Ly9kaXNjb2dz/LWltYWdlcy9SLTk4/OTI5MTItMTU3OTQ1/NjcwNy0yMzIwLmpw/ZWc.jpeg (from here). The webp and jpeg URLs are now the same in their API responses. The URL doesn't seem to include the release ID any longer so maximising a direct image URL no longer works. The real file name is present in the Content-Disposition header of the HTTP response, though, so that could be used instead. Edit: Or just base64-decode the parts after the width (w:) param, excluding slashes. Playing with the size parameters just leads to a 403.

Edited to add: Judging by the new URL format, they probably have switched to using imgproxy over Thumbor. Compare the URLs in the examples below to the one at the bottom of imgproxy's documentation. In practice, this doesn't really affect the viability of grabbing full-size images through this endpoint, since both services use URL signatures (the seemingly random gibberish at the start of the path) to prevent tampering with the parameters. imgproxy uses HMAC-SHA-256 vs. Thumbor's HMAC-SHA-1 though, so there's even more entropy now. They're also using AWS S3 as a storage backend (not new information). It might be worth noting that the old Thumbor URLs currently still work and are redirecting to the imgproxy ones.

TL;DR: Discogs really doesn't want anyone to be able to grab full-size images.

Some examples:

@qsniyg
Copy link
Owner

qsniyg commented Jun 30, 2023

Another interesting piece of information, using one of @ROpdebee's links as an example:

https://i.discogs.com/_xL4yC-gjc-awYVWmO4dDmOv-Za3oICJweuYqpnEzwk/rs:fit/g:sm/q:40/h:300/w:300/czM6Ly9kaXNjb2dz/LWltYWdlcy9SLTk4/OTI5MTItMTU3OTQ1/NjcwNy0yMzIwLmpw/ZWc.jpeg

> "czM6Ly9kaXNjb2dz/LWltYWdlcy9SLTk4/OTI5MTItMTU3OTQ1/NjcwNy0yMzIwLmpw/ZWc".split("/").map(x=>atob(x)).join("")
's3://discogs-images/R-9892912-1579456707-2320.jpeg'

Another image:

https://i.discogs.com/GxhlKZ2FzWY_oT8irrZ8AlA-jQXnnHYefaxL6KKffT4/rs:fit/g:sm/q:40/h:300/w:300/czM6Ly9kaXNjb2dz/LWRhdGFiYXNlLWlt/YWdlcy9SLTIwMTcz/NDM4LTE2NTUyNjY3/NDgtMzUzMy5qcGVn.jpeg

Returns this:

s3://discogs-database-images/R-20173438-1655266748-3533.jpeg

discogs-database-images is a valid S3 bucket (discogs-database-images.s3.amazonaws.com).

@virtualcrane
Copy link

is there any update on a fix for this?

@qsniyg
Copy link
Owner

qsniyg commented Oct 25, 2023

@virtualcrane None so far, sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impossible? Websites that might be impossible to support topic:rules
Projects
None yet
Development

No branches or pull requests

6 participants