First call of CSV.File very slow

[ufechner7]

I have the following code:

@time using CSV

const input="""
time,ping
1,25.7
2,31.8
"""

function read_csv(in)
    io = IOBuffer(in)
    @time file = CSV.File(io)
    close(io)
    file
end

file = read_csv(input)

Which give the following output:

julia> @time include("bench4.jl")
  2.255220 seconds (5.69 M allocations: 328.290 MiB, 6.23% gc time, 91.37% compilation time)
 11.489286 seconds (2.32 M allocations: 96.562 MiB, 99.98% compilation time)
 17.222498 seconds (53.52 M allocations: 2.279 GiB, 4.79% gc time, 98.84% compilation time)
2-element CSV.File:
 CSV.Row: (time = 1, ping = 25.7)
 CSV.Row: (time = 2, ping = 31.8)

julia> versioninfo()
Julia Version 1.7.1
Commit ac5cc99908 (2021-12-22 19:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

So the first call of this toy example needs 11.5 s on a pretty fast machine.

I consider this a bug, it should be possible to bring this time down by a factor of 2 or 3 just by using
proper pre-compilation. Am I missing something?

[rafaqz]

Type instability pretty much everywhere in very large functions. There are easy improvements everywhere just by adding function barriers and splitting things up.

For example, moving the ctx.threaded code into another function from file.jl line 229 along shave the time from 16 to 12 second on my laptop. There is no need to compile that code in single threaded contexts.

[ufechner7]

The following code is about 10 times as fast for this toy example:

@time using DataFrames, DelimitedFiles

const input="""
time,ping
1,25.7
2,31.8
"""

function read_csv(inp)
    @time data, header = readdlm(IOBuffer(inp), ',',header=true)
    # @time df = DataFrame(data, vec(header))
    @time df = identity.(DataFrame(ifelse.(data .== "", missing, data), vec(header)))
    @time df[!,:time] = convert.(Int64,df[:,:time])
    df
end

df = read_csv(input)

It also handles missing values and different column types correctly.

So Julia CAN be reasonably fast with parsing small CSV files. And what does small mean? Even with 50000 entries the code above is still pretty fast (4s instead of 19s that CSV.jl needs).

[ufechner7]

Nice improvements with Julia 1.9-beta2:

@time @eval include("bench4.jl")

# Julia 1.8.4:     7.68s
# Julia 1.9-beta2: 0.59s

So I guess we can close this when 1.9 is released.