write(io, ::Int) allocates

[Seelengrab]

julia> p, io = mktemp()
("/tmp/jl_ry8Ly0", IOStream(<fd 21>))

julia> @time write(io, 1234)
  0.000006 seconds (1 allocation: 16 bytes)
8

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_PKG_SERVER =
  JULIA_NUM_THREADS = 4

Called in a tight loop, this quickly racks up allocations, overwhelming GC. This seems to be caused by the Int getting wrapped in a Ref further down the call chain, forcing it to be moved to the heap because of a @no_inline even further down the call chain.

Feels like primitve types in write shouldn't allocate.

[kwdye]

To give a little more color to the issue, here's an example of the GC becoming "overwhelmed"

function write_iobs(iobs, data)
    Threads.@threads for ii in 1:length(iobs)
        for jj in 1:length(data)
            write(iobs[ii], data[jj])
        end
    end
end

let
    num_ints = 10000000
    num_threads = 128

    data = rand(Int, num_ints)
    iobs = map(x->IOBuffer(Vector{UInt8}(undef, sizeof(Int) * num_ints), write=true), 1:num_threads)

    @time write_iobs(iobs, data)

    iobs
end

for me gives output:

 33.249133 seconds (1.28 G allocations: 19.078 GiB, 94.82% gc time)

Normally, when there are few threads, these tiny allocations are handled fine and you may not even notice. When there's a lot of them, however, it seems like the normal logic in the GC for handling tiny short lived allocations breaks. I think it may be related to this issue:

a8eaa22

[kwdye]

Any IO object that has its own internal buffer perhaps should be encouraged to implement something like this:

function write(io::IOBuffer, v::T) where {T <: Union{Int16,UInt16,Int32,UInt32,Int64,UInt64,Int128,UInt128,Float16,Float32,Float64}}
    re = reinterpret(T, io.data)
    re[1 + div((io.ptr - 1), sizeof(T))] = v
    io.ptr += sizeof(T)
    io.size += sizeof(T)
    return sizeof(T)
end

and it won't allocate. Really this would work for any immutable. TranscodingStreams.jl in particular would benefit from this.

Edit: Modulo the alignment issues when writing different length types!

[mgkuhn]

Could the current allocating implementation

@noinline unsafe_write(s::IO, p::Ref{T}, n::Integer) where {T} =
    unsafe_write(s, unsafe_convert(Ref{T}, p)::Ptr, n) # mark noinline to ensure ref is gc-rooted somewhere (by the caller)
unsafe_write(s::IO, p::Ptr, n::Integer) = unsafe_write(s, convert(Ptr{UInt8}, p), convert(UInt, n))
write(s::IO, x::Ref{T}) where {T} = unsafe_write(s, x, Core.sizeof(T))
function write(s::IO, x::Union{Int16,UInt16,Int32,UInt32,Int64,UInt64,
                               Int128,UInt128,Float16,Float32,Float64})
    return write(s, Ref(x))
end

in base/io.jl perhaps be replaced with something along the lines of

[...]
function write(io::IO, v::T) where {T}
    if isbitstype(T)
        p = unsafe_convert(Ptr{UInt8}, v)
        return unsafe_write(io, p, Core.sizeof(T))
    else
        error("`write` is not supported on non-isbits type ", string(T))
    end
end

Then it wouldn't be necessary any more to wrap “user-defined plain-data types without write methods” into a Ref, as is currently suggested by the docstring of write.

(Although, unsafe_convert wasn't really meant for that.)

[Seelengrab]

Sounds good to me. Maybe we should add

User-defined non-isbits types should define their own write methods.

to the error message as well.

Maybe an ArgumentError would also be better than plain error?

[simeonschaub]

It's not possible to just convert a bitstype to a pointer like this, you need to put it in some kind of mutable container to get a stable memory address. Maybe we can reuse the same Ref, but I am not sure how difficult that would be to make thread safe. I also wouldn't be surprised if our GC was already optimized for small allocations like this, so it might be good to benchmark the benefit of reusing the Ref first.

[Seelengrab]

Hmm.. I get the reasoning for general isbits types, but primitive types really shouldn't have to be wrapped in a Ref in the first place, right? Aren't they alwas stored inline?

[simeonschaub]

Aren't they alwas stored inline?

Not always, but most of the time. That's exactly the problem, since that means it's not possible to get a stable pointer to them, but the way pipes work, the OS requires a pointer to the data. I don't think there's really any way around that requirement.

[kwdye]

Is there a way to perhaps create something that can iterate the bytes of a bitstype? Its easy to do with an Int or Float etc using bitshifts, so I guess this could be recursively done on simple structs as well. The issue would be handling holes and other alignment issues.

[simeonschaub]

You still need a pointer one way or another

[kwdye]

What I mean is you only need the value of an Int to iterate all 8 of its bytes, you just shift and mask and cast the value.

[simeonschaub]

Sure, but unsafe_write is not just like a generic Julia function since it interacts with the OS, so it needs a pointer not an arbitrary iterator.

[kwdye]

But many of the cases where unsafe_write is being called, it doesnt end up writing to a system pipe, e.g., the initial example here. All its doing is writing some bytes to a buffer in Julia space, right?

[simeonschaub]

Yes, IOBuffer could potentially be special-cased here