Memory leak in multi-threaded CSV.read

[bkamins]

I do not remember if it was reported before (as similar issues were reported), but reading the file instagram_posts.csv that can be found in https://www.kaggle.com/datasets/shmalex/instagram-dataset using 4 threads leaves 15GB memory leak (after destroying all the visible variables that reference to the read file) + GC.gc() is very slow all the time.

When doing the same on a single thread all is OK, i.e. after removing references to the read file and doing GC.gc memory is back to the previous level.

Configuration: Win11, Julia 1.8.2, CSV.jl 0.10.4.

[quinnj]

@bkamins, can you check your script/workflow on #1046? I believe we're probably also running into a similar issue in Arrow.jl w/ multithreaded reading/writing. It's probably also worth considering for DataFrames.jl and any other packages utilizing Threads.@spawn.

[bkamins]

#1046 resolves the issue.

CC @nalimilan regarding Threads.@spawn in DataFrames.jl