Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extractor for 500px is broken #1335

Closed
no95typem opened this issue Feb 22, 2021 · 2 comments
Closed

Extractor for 500px is broken #1335

no95typem opened this issue Feb 22, 2021 · 2 comments

Comments

@no95typem
Copy link

When I trying to download any gallery from 500px the error is occurred.
Log:

~ >>> gallery-dl --verbose https://500px.com/p/Light_Expression_Photography/galleries/street                                                                                                                               [1]
[gallery-dl][debug] Version 1.14.5
[gallery-dl][debug] Python 3.9.1 - Linux-5.9.16-1-MANJARO-x86_64-with-glibc2.33
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.3
[gallery-dl][debug] Starting DownloadJob for 'https://500px.com/p/Light_Expression_Photography/galleries/street'
[500px][debug] Using _500pxGalleryExtractor for 'https://500px.com/p/Light_Expression_Photography/galleries/street'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api.500px.com:443
[urllib3.connectionpool][debug] https://api.500px.com:443 "GET /graphql?operationName=ProfileRendererQuery&variables=%7B%22username%22%3A+%22Light_Expression_Photography%22%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22db1dba2cb7b7e94916d1005db16fea1a39d6211437b691c4de2f1a606c21c5fb%22%7D%7D HTTP/1.1" 200 100
[500px][error] An unexpected error occurred: KeyError - 'data'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[500px][debug] 
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.9/site-packages/gallery_dl/job.py", line 64, in run
    for msg in self.extractor:
  File "/home/user/.local/lib/python3.9/site-packages/gallery_dl/extractor/500px.py", line 32, in items
    data = self.metadata()
  File "/home/user/.local/lib/python3.9/site-packages/gallery_dl/extractor/500px.py", line 154, in metadata
    user = self._request_graphql(
  File "/home/user/.local/lib/python3.9/site-packages/gallery_dl/extractor/500px.py", line 87, in _request_graphql
    return self.request(url, params=params).json()["data"]
KeyError: 'data'
@no95typem
Copy link
Author

User profiles downloading doesn't work too

@no95typem no95typem changed the title Extractor for 500px galleries is broken Extractor for 500px is broken Feb 23, 2021
@mikf
Copy link
Owner

mikf commented Feb 24, 2021

You will have more success with downloading from 500px.com in general by using the latest version (1.16.5) instead of 1.14.5, but photos in that gallery in particular cause a different problem: https://api.500px.com/v1/photos refuses to return any info for those photos for some reason.

gallery-dl -v https://500px.com/p/Light_Expression_Photography/galleries/street
[gallery-dl][debug] Version 1.17.0-dev - Git HEAD: 92071d02
[gallery-dl][debug] Python 3.9.1 - Linux-5.4.98-1-lts-x86_64-with-glibc2.33
[gallery-dl][debug] requests 2.25.1 - urllib3 1.25.11
[gallery-dl][debug] Starting DownloadJob for 'https://500px.com/p/Light_Expression_Photography/galleries/street'
[500px][debug] Using _500pxGalleryExtractor for 'https://500px.com/p/Light_Expression_Photography/galleries/street'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api.500px.com:443
[urllib3.connectionpool][debug] https://api.500px.com:443 "GET /graphql?operationName=ProfileRendererQuery&variables=%7B%22username%22%3A+%22Light_Expression_Photography%22%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22105058632482dd2786fd5775745908dc928f537b28e28356b076522757d65c19%22%7D%7D HTTP/1.1" 200 2109
[urllib3.connectionpool][debug] https://api.500px.com:443 "GET /graphql?operationName=GalleriesDetailQueryRendererQuery&variables=%7B%22galleryOwnerLegacyId%22%3A+%225797936%22%2C+%22ownerLegacyId%22%3A+%225797936%22%2C+%22slug%22%3A+%22street%22%2C+%22token%22%3A+null%2C+%22pageSize%22%3A+20%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22fb8bb66d31b58903e2f01ebe66bbe7937b982753be3211855b7bce4e286c1a49%22%7D%7D HTTP/1.1" 200 8061
[urllib3.connectionpool][debug] https://api.500px.com:443 "GET /v1/photos?expanded_user_info=true&include_tags=true&include_geo=true&include_equipment_info=true&vendor_photos=true&include_licensing=true&include_releases=true&liked_by=1&following_sample=100&image_size=4096&ids=1028145219%2C1028145220%2C1028180020%2C1028180087%2C1028180400 HTTP/1.1" 200 None
[500px][error] An unexpected error occurred: KeyError - '1028145219'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[500px][debug] 
Traceback (most recent call last):
  File "/home/mike/dev/python/gallery_dl/job.py", line 69, in run
    for msg in self.extractor:
  File "/home/mike/dev/python/gallery_dl/extractor/500px.py", line 35, in items
    for photo in self.photos():
  File "/home/mike/dev/python/gallery_dl/extractor/500px.py", line 189, in photos
    yield from self._extend(photos["edges"])
  File "/home/mike/dev/python/gallery_dl/extractor/500px.py", line 70, in _extend
    return [
  File "/home/mike/dev/python/gallery_dl/extractor/500px.py", line 71, in <listcomp>
    data[str(edge["node"]["legacyId"])]
KeyError: '1028145219'

mikf added a commit that referenced this issue Mar 4, 2021
instead of crashing with a KeyError exception
@mikf mikf closed this as completed Dec 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants