Fetch all on large tables

[thanuj11]

Hello,
I am trying to fetch 4 million records from an oracle table and convert it into a pandas dataframe. I tried using fetchall() method but it is very slow for some reason and not returning any result.

               cursor.arraysize=5000
                **statement  = f"""
                SELECT * from sales
                """
                print(statement)
                cursor.execute(statement)
                res= cursor.fetchall() **

This table has around 4.5 million rows, I want to fetch all the rows and convert them to dataframe.

When I run a similar query with rows <200k, it returns the result in 2-5 seconds but when I run the query on tables with large data then I didn't get any results back.

@cjbj , Is there other ways I can try to fetch millions of records?

Additional info
Python Version: 3.6
Cx_Oracle version: 7.3.0

[cjbj]

@thanuj11 I'm not a pandas user. I think I've seen similar questions on StackOverflow. It sounds like your machine can't cope with that much data and you need to fetch smaller batches.

[cjbj]

Closing - no activity