Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

484/zenodo integration #1257

Merged
merged 5 commits into from Oct 27, 2022
Merged

484/zenodo integration #1257

merged 5 commits into from Oct 27, 2022

Conversation

shashigharti
Copy link
Contributor

@shashigharti shashigharti commented Sep 21, 2022

  • Read/write files from zenodo
  • Create catalog from single or multiple repositories
  • Added tests
  • Added read/write function to package
  • Made changes to control class and plugin class
  • fixes Integration with Zenodo as a data portal #484
  • Search by status was not working as expected so I will report it.

Please make sure that all the checks pass. Please add here any additional information regarding this pull request. It's highly recommended that you link this PR to an issue (please create one if it doesn't exist for this PR)

Added read/write/catalog function
Added tests for all 3 functions
Fixed code in plugin and control class
Added functions read/write to package class
Fixed formatting of test datapackage file for ndjson
Filter token key from url in vcr
@shashigharti
Copy link
Contributor Author

@roll this is ready for review. Thanks!

@roll roll requested a review from aivuk October 3, 2022 05:57
try:
package = Package.from_zip(path, title=title)
except FrictionlessException as exception:
if not "[Errno 2] No such file or directory" in str(exception):
Copy link
Contributor

@aivuk aivuk Oct 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the error message "No such file or directory" the best one from when the zippeds package fails to be loaded?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is raised when from_zip function doesn't find "datapackage.json" file after it unzips the folder and searches for 'datapackage.json' or 'datapackage.yaml' file in unzipped folder.

https://github.com/frictionlessdata/framework/blob/main/frictionless/package/package.py#L838
https://github.com/frictionlessdata/framework/blob/main/frictionless/package/package.py#L154

So this is to skip the error and continue reading files if it can't read the zipped folder. But I am not sure if this is the right way to skip the exception.

Need suggestion on it @aivuk @roll

@shashigharti
Copy link
Contributor Author

Thank you @aivuk!.

I have added the comments and also replied in PR.

I have made new changes it is ready for review! @roll @aivuk

Copy link
Member

@roll roll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Amazing work!

Of course, we need docs also - https://framework.frictionlessdata.io/docs/portals/zenodo.html. Can you please add a separate PR for it? (also a small blog post I think)

@roll roll merged commit 245649e into main Oct 27, 2022
@roll roll deleted the 484/zenodo-integration branch October 27, 2022 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Integration with Zenodo as a data portal
3 participants