Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: document read_csv fastpath for iso8601 datetime strings #5993

Merged
1 commit merged into from Jan 18, 2014
Merged

DOC: document read_csv fastpath for iso8601 datetime strings #5993

1 commit merged into from Jan 18, 2014

Conversation

ghost
Copy link

@ghost ghost commented Jan 18, 2014

While investigating an optimization suggested in #5490 , I came
across a fast path in pd.read_csv that I was unaware of for parsing iso8601 datetimes.
It's undocumented as far as I can tell.

RIght the wrong.

N=10000
r=pd.date_range("2000-01-01","2013-01-01",freq="H")[:N]
df=pd.DataFrame(range(N),index=r)
df.to_csv("/tmp/1.csv")
def f1(s):
    return dt.datetime.strptime(s,"%Y-%m-%d %H:%M:%S")
def f2(s):
    return dateutil.parser.parse(s)

%timeit x1=pd.read_csv("/tmp/1.csv",parse_dates=True,index_col=0) # fast-path for iso8601, fallback to dateutil.parser
%timeit x2=pd.read_csv("/tmp/1.csv",parse_dates=True,index_col=0,date_parser=f1) 
%timeit x2=pd.read_csv("/tmp/1.csv",parse_dates=True,index_col=0,date_parser=f2)

100 loops, best of 3: 14.1 ms per loop
1 loops, best of 3: 259 ms per loop
1 loops, best of 3: 956 ms per loop

ghost pushed a commit that referenced this pull request Jan 18, 2014
DOC: document read_csv fastpath for iso8601 datetime strings
@ghost ghost merged commit e2a612a into pandas-dev:master Jan 18, 2014
@ghost ghost deleted the PR_doc_iso8601_fast_path branch January 18, 2014 20:07
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

0 participants