Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
PERF: Removed the GIL from parts of the TextReader class #11272
The GIL was removed around the tokenizer functions and the conversion function(_string_convert excluded).
import pandas as pd import numpy as np df = pd.DataFrame(np.random.randn(1000000,10)) df.to_csv('test.csv')
import pandas as pd from pandas.util.testing import test_parallel def f(): for i in range(4): pd.read_csv('test.csv', index_col=0) @test_parallel(4) def g(): pd.read_csv('test.csv', index_col=0)
In : %timeit pd.read_csv('test.csv', index_col=0) 1 loops, best of 3: 2.35 s per loop In : %timeit f() 1 loops, best of 3: 9.55 s per loop In : %timeit g() 1 loops, best of 3: 4.38 s per loop
some windows cythoning errors.
odd they don't show up for you, what platform are you testing on?
builds ,couple of errors on windows