Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[load][xs]: allow setting encoding #40

Closed
wants to merge 1 commit into from
Closed

[load][xs]: allow setting encoding #40

wants to merge 1 commit into from

Conversation

zelima
Copy link

@zelima zelima commented Oct 18, 2018

I have a source that datapackage-py cannot detect encoding for. And getting errors when trying run flows. PR allows setting encoding for resources if that is present in the descriptor

Flow(
    load(
        load_source='https://www2.census.gov/programs-surveys/cps/tables/time-series/historical-income-households/h01ar.xls',
        format='xls',
        sheet= 1,
        skip_rows=list(range(1,62)) + [-1],
        encoding='utf-8',
        headers=['Year', 'Number (thousands)', 'Lowest', 'Second', 'Third', 'Fourth', 'Top 5 percent'],
    ),
    printer()
).process()

Error message

  File "/home/zelima/.virtualenvs/data-factory/lib/python3.6/site-packages/dataflows/processors/load.py", line 55, in process_datapackage
    self.res.infer(confidence=1, limit=1000)
  File "/home/zelima/.virtualenvs/data-factory/lib/python3.6/site-packages/datapackage/resource.py", line 256, in infer
    encoding = cchardet.detect(contents)['encoding'].lower()
AttributeError: 'NoneType' object has no attribute 'lower'

@zelima zelima requested a review from akariv October 18, 2018 12:11
@coveralls
Copy link

coveralls commented Oct 18, 2018

Pull Request Test Coverage Report for Build 191

  • 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.02%) to 77.747%

Totals Coverage Status
Change from base Build 187: 0.02%
Covered Lines: 1118
Relevant Lines: 1438

💛 - Coveralls

@akariv
Copy link
Member

akariv commented Oct 18, 2018

@zelima
This bug was fixed in datapackage-py and deployed this morning.
lmk if it remains once you update datapackage and tableschema libs.

@zelima
Copy link
Author

zelima commented Oct 18, 2018

@akariv this issue described above is fixed for me right now datapackage-py>=1.5.1. I assume we can close this one

@zelima zelima closed this Oct 18, 2018
@zelima zelima deleted the fix/encoding branch October 18, 2018 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants