Fix to dataset extractor #640
Conversation
i think this fixes the issue with file upload, but not sure whether it will cause any other changes.
uploaded by mmistake
…dataset-extractor # Conflicts: # frontend/package-lock.json
|
So the key issue is "token" no longer needed since it's handled by dependencies? |
|
Yes. There was a dependency for Security that we no longer use that only works with token, but not API key. It was removed in other routes but was still in the file upload. Token was used in methods but, in the end, wasn't used in the last one, so I removed it. The user info is still used with rabbitmq and posting the jobs. |
tcnichol
left a comment
tcnichol
left a comment
There was a problem hiding this comment.
Tested.
Works with the pyclowder branch, the file posts into the dataset now. Approved.
… with the method from mimetypes also changed in visualization.
|
I made 1 fix here based on the debugging from Friday. Mimetypes.get_content_type was being given a io.FileBytesIO, but it needed the file name. So it now is sent the file name from the new_file File object that is created. I tested and this still works with visualization. Since I had it working I went ahead and pushed here while I remembered. With the dataset extractor, the file that gets sent does have None for a filetype but this does correctly find the filetype from the filename and the entry is added in minio. When downloaded the file is the same now. |
|
@tcnichol Thanks. It needed one little change. Now everything is working. |
max-zilla
left a comment
max-zilla
left a comment
There was a problem hiding this comment.
tested with pyclowder PR extractor, merging both
fixes to dataset.py to make dataset extractor working.
To test, please run test-dataset-extractor from clowder-framework/pyclowder#67
Trigger it on any dataset.