Skip to content

Commit

Permalink
truncate excess output per execution request
Browse files Browse the repository at this point in the history
  • Loading branch information
tlnagy committed Sep 15, 2016
1 parent bbfd419 commit d42fa06
Showing 1 changed file with 26 additions and 5 deletions.
31 changes: 26 additions & 5 deletions src/stdio.jl
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,41 @@ end
const bufs = Dict{String,IOBuffer}()
const stream_interval = 0.1
const max_bytes = 10*1024
# max output per code cell is 512 kb
const max_output_per_request = 1 << 19

"""Continually read from (size limited) Libuv/OS buffer into an (effectively unlimited) `IObuffer`
to avoid problems when the Libuv/OS buffer gets full (https://github.com/JuliaLang/julia/issues/8789).
Send data immediately when buffer contains more than `max_bytes` bytes. Otherwise, if data is available
it will be sent every `stream_interval` seconds (see the Timers set up in watch_stdio)"""
"""
Continually read from (size limited) Libuv/OS buffer into an `IObuffer` to avoid problems when
the Libuv/OS buffer gets full (https://github.com/JuliaLang/julia/issues/8789). Send data immediately
when buffer contains more than `max_bytes` bytes. Otherwise, if data is available it will be sent every
`stream_interval` seconds (see the Timers set up in watch_stdio). Truncate the output to `max_output_per_request`
bytes per execution request since excessive output can bring browsers to a grinding halt.
"""
function watch_stream(rd::IO, name::AbstractString)
task_local_storage(:IJulia_task, "read $name task")
try
buf = IOBuffer()
bufs[name] = buf
counter = 0
curr_msg = ""
while !eof(rd) # blocks until something is available
nb = nb_available(rd)
parent_id_exists = "msg_id" in keys(execute_msg.header)
continuation = curr_msg == execute_msg.header["msg_id"]
# if this a new stream then reset the byte counter
if parent_id_exists && !continuation
curr_msg = execute_msg.header["msg_id"]
counter = 0
end
if nb > 0
write(buf, read(rd, nb))
counter += nb
# if this stream as surpassed the maximum output limit then ignore future bytes
if parent_id_exists && continuation && counter > max_output_per_request
(counter - nb <= max_output_per_request) && println(orig_STDOUT[], "Excessive output truncated.")
read(rd, nb) # read from libuv/os buffer and throw data away
else
write(buf, read(rd, nb))
end
end
if buf.size > 0
if buf.size >= max_bytes
Expand Down

0 comments on commit d42fa06

Please sign in to comment.