New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor CSV has_header inference #192

Merged
merged 13 commits into from May 8, 2015

Conversation

Projects
None yet
2 participants
@cpcloud
Member

cpcloud commented May 7, 2015

No description provided.

@cpcloud cpcloud self-assigned this May 7, 2015

@cpcloud cpcloud added this to the 0.3.3 milestone May 7, 2015

@cpcloud cpcloud added the enhancement label May 7, 2015

cpcloud added some commits May 7, 2015

@cpcloud

This comment has been minimized.

Member

cpcloud commented May 7, 2015

@mrocklin this could use a bit of review when you get a chance

@cpcloud

This comment has been minimized.

Member

cpcloud commented May 7, 2015

things are passing on win64 py27 and py34 as well as osx py27 and py34

def infer_header(path, nbytes=10000, encoding='utf-8', **kwargs):
with open_file(path, 'rb') as f:
raw = f.read(nbytes)
return csv.Sniffer().has_header(raw if PY26 else raw.decode(encoding))

This comment has been minimized.

@mrocklin

mrocklin May 7, 2015

Member

I was burned sufficiently often by windows and py2/3 differences to avoid the csv module alltogether in favor of pandas.read_csv. If you've handled that then that sounds good.

Curious, what triggered the move away here?

This comment has been minimized.

@cpcloud

cpcloud May 7, 2015

Member

It wasn't working automatically for tab separated files

cpcloud added some commits May 7, 2015

cpcloud added a commit that referenced this pull request May 8, 2015

Merge pull request #192 from cpcloud/csv-fixes
Refactor CSV has_header inference

@cpcloud cpcloud merged commit 574e089 into blaze:master May 8, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@cpcloud cpcloud deleted the cpcloud:csv-fixes branch May 8, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment