Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataPackage viewer: does not support csv dialect #16

Closed
sirex opened this issue May 5, 2016 · 5 comments
Closed

DataPackage viewer: does not support csv dialect #16

sirex opened this issue May 5, 2016 · 5 comments

Comments

@sirex
Copy link

sirex commented May 5, 2016

I have following resource definition:

{
    "url": "http://gobetween.oklabs.org/pipe/https%3A%2F%2Fraw.githubusercontent.com%2Fvilnius%2Ftransportas-velavimai%2Fmaster%2Frpt3%2Flatest.tsv",
    "format": "csv",
    "mediatype": "text/csv",
    "dialect": {
        "delimiter": "\t"
    }
}

But it looks, like DataPackage viewer ignores dialect. I tried same thing with Python datapackage lib, it parses data correctly.

Full example can be found here:
http://data.okfn.org/tools/view?url=https%3A%2F%2Fraw.githubusercontent.com%2Fsirex%2Ftransportas-velavimai%2Fmaster%2Fdatapackage.json#readme

And full datapackage.json can be found here:
https://github.com/sirex/transportas-velavimai/blob/master/datapackage.json

@rufuspollock
Copy link
Contributor

@sirex you are quite right that this needed to be fixed - see existing TODO here https://github.com/frictionlessdata/datapackage-render-js/blob/master/datapackage.js#L143

This is actually a 10m job as all we need to do is convert arguments over to the ones that csv lib uses. If you are up for it PRs are very welcome.

@rufuspollock
Copy link
Contributor

@sirex if you pull master now there should be delimiter support in. We are still working to support the rest of the dialect.

@rufuspollock
Copy link
Contributor

References:

rufuspollock added a commit that referenced this issue May 7, 2016
[#16,dp][m]: proper support for delimiter, quoteChar and escapeChar in csv dialect.
@zelima
Copy link
Contributor

zelima commented May 12, 2016

CSV dialect specifications mapping status with CSV Parser Options

  • delimiter - mapped with delimiter
  • lineTerminator -mapped with rowDelimiter
  • quoteChar - mapped with quote
  • skipInitialSpace - mapped with ltrim [updated]
  • header - mapped with columns
  • escapeChar - not mapped (not supported)
  • doubleQuote - mapped with escape = '', when set to false [update]
  • nullSequence - not mapped (not supported)
  • caseSensitiveHeader - not mapped (not supported)

Update

@anuveyatsu
Copy link
Member

Now irrelevant due to the library has changed - now focusing on utility functions for compiling and converting datapackage view specifications

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants