Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use full url to download files. #283

Merged
merged 6 commits into from May 3, 2023
Merged

Use full url to download files. #283

merged 6 commits into from May 3, 2023

Conversation

crisely09
Copy link
Contributor

Related to #282
The idea is to check if the contentUrl is already expanded and use it as given.
When the file_id is passed, then do the standard url construction, i.e. endpoint/org/proj/file_id .

@codecov-commenter
Copy link

codecov-commenter commented Apr 7, 2023

Codecov Report

Merging #283 (c992e76) into master (e4edeca) will decrease coverage by 0.10%.
The diff coverage is 8.69%.

@@            Coverage Diff             @@
##           master     #283      +/-   ##
==========================================
- Coverage   72.85%   72.75%   -0.10%     
==========================================
  Files          88       88              
  Lines        5574     5583       +9     
==========================================
+ Hits         4061     4062       +1     
- Misses       1513     1521       +8     
Flag Coverage Δ
unittests 72.75% <8.69%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
kgforge/specializations/resources/datasets.py 62.88% <0.00%> (ø)
kgforge/specializations/stores/bluebrain_nexus.py 22.43% <7.69%> (+0.09%) ⬆️
kgforge/core/archetypes/store.py 58.87% <12.50%> (-2.70%) ⬇️

@MFSY
Copy link
Collaborator

MFSY commented Apr 9, 2023

Looks good. Fixed the following errors (introduced in previous merged commits) occurring when downloading:

  • Before, _download_many were failing if at least one resource did not have a requested content_type even if other resources in the list had. Now, only resources with the provided content_type are considered when downloading
  • Before, Dataset.download() was raising a ValueError when 'follow' was provided because "source" argument has a default value and providing 'source' and 'follow' arguments was not allowed. Now, warnings are raised (asking to choose one of the two and stating that 'source' will be deprecated) and 'follow' is always used when provided.

@crisely09
Copy link
Contributor Author

I see that you added the solution to the cross_bucket problem here, that is good, two birds in one shot. Maybe we could add an example for downloading using cross_bucket=True if it doesn't already exist.

@MFSY
Copy link
Collaborator

MFSY commented May 3, 2023

I see that you added the solution to the cross_bucket problem here, that is good, two birds in one shot. Maybe we could add an example for downloading using cross_bucket=True if it doesn't already exist.

It is mentioned both in the Querying and Dataset notebooks.

@MFSY MFSY merged commit ffbed31 into master May 3, 2023
1 check passed
@MFSY MFSY deleted the fix_download branch May 3, 2023 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants