BoundsError writing a simple CSV

[yurivish]

julia> using CSV

julia> s = join(1:1000000);

julia> length(s)
5888896

julia> CSV.write("out.test.csv", [(a=s,)])
ERROR: BoundsError: attempt to access 4194304-element Array{UInt8,1} at index [1:5888896]
Stacktrace:
 [1] throw_boundserror(::Array{UInt8,1}, ::Tuple{UnitRange{Int64}}) at ./abstractarray.jl:537
 [2] checkbounds at ./abstractarray.jl:502 [inlined]
 [3] view at ./subarray.jl:163 [inlined]
 [4] writedelimnewline(::Array{UInt8,1}, ::Int64, ::Int64, ::IOStream, ::UInt8) at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:248
 [5] #82 at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:303 [inlined]
 [6] macro expansion at /Users/yurivish/.julia/packages/Tables/okt7x/src/utils.jl:71 [inlined]
 [7] eachcolumn at /Users/yurivish/.julia/packages/Tables/okt7x/src/utils.jl:66 [inlined]
 [8] writerow at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:298 [inlined]
 [9] #73 at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:177 [inlined]
 [10] (::CSV.var"#80#81"{CSV.var"#73#74"{Bool,Bool,Tables.Schema{(:a,),Tuple{String}},Array{NamedTuple{(:a,),Tuple{String}},1},CSV.Options{UInt8,UInt8,Nothing,Tuple{},CSV.var"#68#70"},Tuple{Symbol},Int64,Int64,Array{UInt8,1}}})(::IOStream) at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:241
 [11] open(::CSV.var"#80#81"{CSV.var"#73#74"{Bool,Bool,Tables.Schema{(:a,),Tuple{String}},Array{NamedTuple{(:a,),Tuple{String}},1},CSV.Options{UInt8,UInt8,Nothing,Tuple{},CSV.var"#68#70"},Tuple{Symbol},Int64,Int64,Array{UInt8,1}}}, ::String, ::Vararg{String,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:298
 [12] open(::Function, ::String, ::String) at ./io.jl:296
 [13] with at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:240 [inlined]
 [14] #write#72 at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:167 [inlined]
 [15] write(::Tables.Schema{(:a,),Tuple{String}}, ::Array{NamedTuple{(:a,),Tuple{String}},1}, ::String, ::CSV.Options{UInt8,UInt8,Nothing,Tuple{},CSV.var"#68#70"}) at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:162
 [16] write(::String, ::Array{NamedTuple{(:a,),Tuple{String}},1}; delim::Char, quotechar::Char, openquotechar::Nothing, closequotechar::Nothing, escapechar::Char, newline::Char, decimal::Char, dateformat::Nothing, quotestrings::Bool, missingstring::String, transform::CSV.var"#68#70", bom::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:154
 [17] write(::String, ::Array{NamedTuple{(:a,),Tuple{String}},1}) at /Users/yurivish/.julia/packages/CSV/vyG0T/src/write.jl:147
 [18] top-level scope at REPL[9]:1

julia> versioninfo()
Julia Version 1.4.0
Commit b8e9a9ecc6 (2020-03-21 16:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

The same example succeeds if you make s a shorter string.

[yurivish]

Oh, I see this is documented to be the case due to a default 4mb buffer size in CSV.write:

CSV.jl/src/write.jl

Line 51 in 18f8e51

* `bufsize::Int=2^22`: The length of the buffer to use when writing each csv-formatted row; default 4MB; if a row is larger than the `bufsize` an error is thrown

The error seemed like an unintended error since it didn't mention that this was due to a too-small buffer size, and I ran into this while trying to reproduce a segfault I was running into earlier, so I assumed this error was a bug.

This issue can be changed into an issue about a better error message here if folks agree that this is reasonable.

[quinnj]

Thanks for the report! I have a fix w/ a more descriptive, helpful error in #647

[yurivish]

Awesome, thank you!