[Python] Pandas to arrow conversion slowed down by local imports

[asfimport]

The pandas to arrow conversion is currently slowed down significantly by various local import statements.

import pandas as pd
import pyarrow as pa
import cProfile
ser = pd.Series(range(10000))
df = pd.DataFrame({col: ser.copy(deep=True) for col in range(50)})
# Simulate a real dataset, i.e. force copy of data
df = df.astype({col: str for col in range(25)})
prof = cProfile.Profile()

prof.enable()
# a few times to collect statistics
for _ in range(100):
    pa.Table.from_pandas(df, nthreads=1)
prof.disable()
prof.dump_stats("array_conversion.prof")

Reporter: Florian Jetter / @fjetter
Assignee: Florian Jetter / @fjetter

Original Issue Attachments:

• image-2019-02-19-19-10-46-330.png

PRs and other links:

• GitHub Pull Request #3706

_{Note: This issue was originally created as ARROW-4629. Please see the migration documentation for further details.}

[asfimport]

Uwe Korn / @xhochy:
Issue resolved by pull request 3706
#3706