-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resource without a format field results in a crash of pipeline #2
Milestone
Comments
Format is deduced from the file extension, or needs to be explicitly
provided. We should make sure the extension is appended to the rawstore
filename when uploading.
…On Thu, Jul 20, 2017, 22:38 Rufus Pollock ***@***.***> wrote:
stream_remote_resources: WARNING :Error while opening resource from url https://s3.amazonaws.com/rawstore-testing.datahub.io/sQqpgDlCdaDFdRjzxbZN9Q==: FormatError('Format "None" is not supported',)
stream_remote_resources: Traceback (most recent call last):
stream_remote_resources: File "/usr/local/lib/python3.6/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 144, in opener
stream_remote_resources: _stream.open()
stream_remote_resources: File "/usr/local/lib/python3.6/site-packages/tabulator/stream.py", line 132, in open
stream_remote_resources: raise exceptions.FormatError(message)
stream_remote_resources: tabulator.exceptions.FormatError: Format "None" is not supported
stream_remote_resources: During handling of the above exception, another exception occurred:
stream_remote_resources: Traceback (most recent call last):
stream_remote_resources: File "/usr/local/lib/python3.6/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 201, in
stream_remote_resources: rows = stream_reader(resource, url, ignore_missing or url == "")
stream_remote_resources: File "/usr/local/lib/python3.6/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 159, in stream_reader
stream_remote_resources: schema, headers, stream, close = get_opener(url, _resource)()
stream_remote_resources: File "/usr/local/lib/python3.6/site-packages/datapackage_pipelines/specs/../lib/stream_remote_resources.py", line 153, in opener
stream_remote_resources: _stream.close()
stream_remote_resources: File "/usr/local/lib/python3.6/site-packages/tabulator/stream.py", line 156, in close
stream_remote_resources: self.__parser.close()
stream_remote_resources: AttributeError: 'NoneType' object has no attribute 'close'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#2>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAQMdX6vQg0J0Q55OT1lalqUaV0TtkABks5sP6zGgaJpZM4Oek2J>
.
|
@akariv but those files had a file extension in the path https://github.com/datasets/country-codes/blob/master/datapackage.json#L44 (note i have now added the format as well). Yet format was not deduced. I guess you are using the file name directly. There are lots of options here e.g.
|
FIXED. We resolved this issue - assembler will now proceed ok if no format. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The text was updated successfully, but these errors were encountered: