You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
astrochun opened this issue
Jun 30, 2021
· 0 comments
· Fixed by #230
Assignees
Labels
apiUse of a third-party APIbugSomething isn't workingcurationPertains to aspects of curation, including workflow managementp1Issues affecting production, multiple users
@jonathannoah identified a weird issue where the files could not be retrieved. I attempted to retrieve it via curl and was successful. Was able to reproduce the error locally with get_data. Ruled out it being a token issue. Today Figshare deployed changes to the API; however, we don't know what changed. This seems to be a urlretrieve issue completely. I think the urllib has not worked well with the Figshare API so I'm thinking we should switch back to requests with a chunking approach.
This is only a problem with under review deposits, not for public datasets/files.
Reproduction Steps
Retrieval via get_data of any file will suffice.
Outputted Messages
13:22:26 - INFO: Writing: ****/1.ToDo/Chun_Ly_7083742/v2/METADATA/file_list_original_7083742.json
13:22:26 - INFO: Writing: ****/1.ToDo/Chun_Ly_7083742/v2/METADATA/file_list_original_7083742.csv
13:22:26 - INFO: Retrieving 1 of 1 : wccfl_logo.jpeg (99241)
13:22:26 - INFO: URL: https://ndownloader.figsh.com/files/829622700
13:22:26 - INFO: Retrieval attempt #1
13:22:28 - WARNING: Caught an HTTPError: HTTP Error 400: Bad Request
13:22:28 - INFO: URL might be public: https://ndownloader.figsh.com/files/829622700
13:22:28 - INFO: Attempting retrieval without token
13:22:29 - WARNING: Caught an HTTPError: HTTP Error 403: Forbidden
13:22:29 - WARNING: Failed to retrieve: ****/1.ToDo/Chun_Ly_7083742/v2/ORIGINAL_DATA/wccfl_logo.jpeg
13:22:29 - INFO: Retrieval attempt #2
13:22:30 - WARNING: Caught an HTTPError: HTTP Error 400: Bad Request
13:22:30 - INFO: URL might be public: https://ndownloader.figsh.com/files/829622700
13:22:30 - INFO: Attempting retrieval without token
13:22:31 - WARNING: Caught an HTTPError: HTTP Error 403: Forbidden
13:22:31 - WARNING: Failed to retrieve: ****/1.ToDo/Chun_Ly_7083742/v2/ORIGINAL_DATA/wccfl_logo.jpeg
13:22:31 - INFO: Retrieval attempt #3
13:22:32 - WARNING: Caught an HTTPError: HTTP Error 400: Bad Request
13:22:32 - INFO: URL might be public: https://ndownloader.figsh.com/files/829622700
13:22:32 - INFO: Attempting retrieval without token
13:22:33 - WARNING: Caught an HTTPError: HTTP Error 403: Forbidden
13:22:33 - WARNING: Failed to retrieve: ****/1.ToDo/Chun_Ly_7083742/v2/ORIGINAL_DATA/wccfl_logo.jpeg
13:22:33 - WARNING: File retrieval unsuccessful! Aborted after 3 tries
The text was updated successfully, but these errors were encountered:
astrochun
added
api
Use of a third-party API
bug
Something isn't working
curation
Pertains to aspects of curation, including workflow management
p1
Issues affecting production, multiple users
labels
Jun 30, 2021
apiUse of a third-party APIbugSomething isn't workingcurationPertains to aspects of curation, including workflow managementp1Issues affecting production, multiple users
Describe the bug
@jonathannoah identified a weird issue where the files could not be retrieved. I attempted to retrieve it via
curl
and was successful. Was able to reproduce the error locally withget_data
. Ruled out it being a token issue. Today Figshare deployed changes to the API; however, we don't know what changed. This seems to be aurlretrieve
issue completely. I think theurllib
has not worked well with the Figshare API so I'm thinking we should switch back torequests
with a chunking approach.This is only a problem with under review deposits, not for public datasets/files.
Reproduction Steps
Retrieval via
get_data
of any file will suffice.Outputted Messages
Version information
Expected behavior
Screenshots
Additional context
Implemented in: TBD
The text was updated successfully, but these errors were encountered: