memory issues

[vbhavsingh]

Does not work with large tables.

19.01.2018 14:08:07 [INF]: converting data to CSV (one dot corresponds with 50k lines).......Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:2694)
at java.lang.String.(String.java:203)
at java.lang.StringBuilder.toString(StringBuilder.java:405)
at sqldump.etl.Dump.toFile(Dump.java:93)
at sqldump.etl.Query.executeQuery(Query.java:221)
at sqldump.etl.Query.getDataFromSQLFile(Query.java:62)
at sqldump.SQLDump.run(SQLDump.java:89)
at sqldump.Main.main(Main.java:7)

[BernhardSchlegel]

How much RAM does your machine have ?

[vbhavsingh]

I changed the code in my fork to dump output to file at every 10k lines, then reset the ArrayList and StringBuffer to save memory.
I was dumping around a 50Gb table and was having around 32Gb RAM.

[BernhardSchlegel]

Would you mind to send me PR ? Any improvement is welcome!

[BernhardSchlegel]

@vbhavsingh ping :) a PR would serve all users and credit you for your work!