Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uploaded file gets corrupted #6

Open
abubelinha opened this issue Feb 5, 2021 · 1 comment
Open

uploaded file gets corrupted #6

abubelinha opened this issue Feb 5, 2021 · 1 comment

Comments

@abubelinha
Copy link

abubelinha commented Feb 5, 2021

Hi. I am a newbie to both Python and Zenodo.

I am trying this library to upload files from Windows 7 (Python 3.4.4) to Zenodo sandbox.
I am doing it like this:

           api = ZenodoHandler(access_token=my_token, proxies={}, test=True)
           filepath = "C:\\Users\\Public\\Pictures\\Sample Pictures\\Tulips.jpg"
           import ntpath
           filename = ntpath.basename(filepath) 
           api.deposition_files_create(deposition_id=725510,
                                        target_name=filename,
                                        file_path=filepath)

OK. It uploads something ... but when I go to web interface, the uploaded file is somehow corrupted.
The web interface says it is much bigger in size (1.6 MB instead of 606 KB, which is the size reported by Windows Explorer).
And if I click on it, of course the Zenodo image preview does not work.


EDIT:
I tried it again with a text file and it gets corrupted as well.

Check out this deposit: https://sandbox.zenodo.org/record/725510

This is the original file content (I uploaded it using the web interface, as samplefile.txt):

I am a text file header
Line 0
Line 1
Line 2

And this is how it gets after uploaded using the script and renamed as sample.txt:

file=I+am+a+text+file+header%0D%0A&file=Line+0%0D%0A&file=Line+1%0D%0A&file=Line+2

So, looks like it is somehow urlencoded during the upload.
How can I avoid this?


RE-EDIT:
I suspect the problem could be related to changes in the API, as I mention in this Zenodo issue.

But as I am a newcomer to both Python and Zenodo, I am not very sure.
Could please someone more experienced compare these lines in different scripts which perform file uploads?

This project's zenodolib.py#L155 (notice the {'file': ... } brackets, where I suspect the problem is):

data = {'file': open(file_path, 'rb')}

@moble sxs/zenodo/api/deposit.py#L682 (NO {'file': ... } brackets):

r = self._put(url, data=open(path, 'rb'), headers={"Content-Type": "application/octet-stream"})

@slint gist #file-zenodo-upload-py-L24-to-L29 (NO {'file': ... } brackets, plus explanation to pass file object directly as 'data'):

# We pass the file object (fp) directly to request as 'data' for stream upload
# the target URL is the URL of the bucket and the desired filename on Zenodo seprated by slash
with open('/path/to/my-file.zip', 'rb') as fp:
    res = requests.put(
        bucket_url + '/my-file.zip', 
        data=fp,
        # No headers included in the request, since it's a raw byte request
        params=params,
    )


EDIT:
Following @moble suggestion, I added a pull request (using the web interface).
First time I do this, sorry if it's not properly done.

abubelinha added a commit to abubelinha/zenodo-python that referenced this issue Feb 6, 2021
correction to avoid corrupted file uploads (due to changes in Zenodo API?)
SiLeBAT#6
abubelinha added a commit to abubelinha/zenodo-python that referenced this issue Feb 6, 2021
adaptation to new file upload API, as commented in SiLeBAT#6
@garciagenrique
Copy link
Contributor

Hi @abubelinha,
I have seen you tag me some days ago.

I think the library is quite deprecated (Zenodo API has evolved a bit, and your PR is pretty much necessary). Because we needed faster interaction, we created our own zenodo-api-library. This library is active (at least until some months ago), so give some days to the main developper to answer ;-).

Also, in my experience, I would suggest to directly email Zenodo people rather than tagging them in issues (they are super friendly and and they answer in few days).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants