Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

RFC: recshow #1139

Closed
wants to merge 4 commits into from

4 participants

@toivoh
Collaborator

A first attempt at a show function that works for self-referential data, per this discussion:
https://groups.google.com/forum/#!topic/julia-dev/qdHQYQb-0XQ

It should work for sets, dicts, composite types that use the default show,
and other types that call rshow rather than show within their own show methods.
I wasn't sure how to get it to work with arrays, since they seem to use some sprintf and showcompact trickery to show themselves.

I've replaced most calls to show within base/, on the premise that rshow should be the user facing function. All of these might not be strictly necessary, but they don't add any overhead to normal shows either.

There's probably still some open issues:

  • Presentation
  • Is it possible to make recshow print incrementally
    rather than consuming the whole object first?
  • Names: show, rshow, recshow
  • Array printing
  • ...

Any feedback appreciated!

@toivoh
Collaborator

Btw, is there a good way to include a test/example script with the pull request?

@toivoh
Collaborator

Also, I should probably make show_composite(io, x) into a staged function, if those are still alive and well.
Good/bad idea?

@pao
Collaborator
pao commented

If its an actual test, add it to the test suite; for an example, I'd pop it in a gist and link that here.

@toivoh
Collaborator

Ok, here's a small example then: https://gist.github.com/3304162

@StefanKarpinski

Here's an interesting idea, regarding showing recursive data structures. If we've already seen something, then there's an expression for it. So we can use that expression as a representation of the thing itself. Example:

julia> x = {};

julia> push(x,x);

julia> x
{x}

This is a hypothetical example — but it would be an ideal way to show this. Even if the shown expression is something fairly complex, this would be possible because Julia is fully expression-oriented, so you can always take an expression that evaluates to something and place it somewhere else. So this would also work:

julia> x[1]
{x[1]}

Of course, this could be better written as {x}, but that would be much harder to implement, requiring a global search of things that are equal to some value — basically the sweep phase of a mark-and-sweep garbage collection, looking for the simplest way to express a specific object. The bigest hurdle to implementing this is the fact that indexing is user-defined, so you can't generally know that writing x[1] is the way to get at the first item in x.

@StefanKarpinski

Also, this approach would clearly require access to the evaluated expression in the repl. That can certainly be arranged, however.

@toivoh
Collaborator

I'm not sure that would suit my purposes, which is typically to inspect programmatically generated graphs from the REPL.

Also, I'd be a lot more comfortable with a general purpose show function that produces a self-contained result.
What if the show result is not going to the REPL, but to a log, or some other destination?

There could be a use for the kind of thing you're describing, for in-REPL experimentation. But then I think it should limit itself to naming objects by global variables that refer to them, so that you can type in the variable and see what it contains.
Still, there will be cases when it's necessary to name an object that has never appeared in the REPL,
so I think any recshow function must be able to refer within it's printout to other parts in the same printout.

@ViralBShah
Owner

What is the latest on this? We certainly need a better way to show self-referential data, but what approach should we take?

@toivoh
Collaborator

I'm not sure that there's been any progress on this.
I think that the major design issue is that it's impossible to achieve simultaneously all of the six desirable features below:

  1. Each (mutable) object printed only once
  2. Each (mutable) object that appears in several places referred to by a common name
  3. Named objects named when they are printed
  4. A nice, readable printout, that isn't cluttered with unnecessary names
  5. Incremental printing: Start printing before the entire graph has been visited
  6. Not going back to erase old printouts/reprinting a fresh one

Basically, this requires to know for each object whether it should be named before it is printed,
which requires to know whether it is referred to by more than one object,
which requires to visit the entire graph first.

We'll need to give up on at least one of these. This pull request gives up number 5.
I'd really like to see a discussion on this design issue.

Besides that, there are a number of decisions to be made (partially dependent on the first answer)

  • What should a printout that contains shared (named) objects look like?
  • How should this be integrated with the current system for show etc?
@toivoh
Collaborator

Jumping ahead a little. As for the visual format, how about something like

julia> a = {}; push(a,a)
(a = {a}; a)

When the graph is acyclic, the output should be able to evaluate to something like the original, just as I think show aims to do now:

julia> a={}; {a,a}
(a={}; {a,a})

(Note: The examples above are mockups, not real julia output)

@ViralBShah
Owner

What are we going to do about this? This pull request has been around for 7 months.

@StefanKarpinski

I was originally very gung-ho about showing everything in a form that would evaluate back to the original version of something, but we ended up not doing that in a lot of places (e.g. displaying arrays), and I think it's better not to. So given that we don't generally do that, it seems fair to give it up. Giving up on property #5 and traversing an entire data structure before printing is probably the best way to go here. Unfortunately, this pull request is now rather old and bringing it up-to-date is likely a fair amount of work (even checking it out and building the state it was once in will take a while, although I might do it on julia.mit.edu with 60 cores or something). @tovioh, can you include an example of the output this produced?

@toivoh
Collaborator

@StefanKarpinski: I agree that giving up on number 5 seems like the best way out.
About the output: is https://gist.github.com/toivoh/3304162 enough?

We should probably have a fresh discussion about the output format. It might very well be that it's easiest to start the implementation afresh as well. Unfortunately, I won't have much time for this for a good while.

@StefanKarpinski

Ok, so it seems like we should close this pull-request in favor of a future one when you get the chance. No worries about the time, but I'm looking forward to it when you do get a chance.

@toivoh
Collaborator
@StefanKarpinski

Closing to be resumed later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 9, 2012
  1. Add recshow.jl to extras/

    Toivo Henningsson authored
  2. Add rshow(io, x) = show(io, x) and make print(io, x) call it.

    Toivo Henningsson authored
  3. Default show for composites from within julia, using rshow.

    Toivo Henningsson authored
  4. Change most calls to show() into rshow(), within base/.

    Toivo Henningsson authored
This page is out of date. Refresh to see the latest.
View
7 base/base.jl
@@ -60,11 +60,14 @@ type ShowError <: Exception
err::Exception
end
-show(io, bt::BackTrace) = show(io,bt.e)
+export rshow
+rshow(io, x) = show(io, x) # hook to allow showing of self-referential data
+
+show(io, bt::BackTrace) = rshow(io,bt.e)
function show(io, se::ShowError)
println("Error showing value of type ", typeof(se.val), ":")
- show(io, se.err)
+ rshow(io, se.err)
end
method_missing(f, args...) = throw(MethodError(f, args))
View
4 base/complex.jl
@@ -17,14 +17,14 @@ reim(z) = (real(z), imag(z))
function _jl_show(io, z::Complex, compact::Bool)
r, i = reim(z)
if isnan(r) || isfinite(i)
- compact ? showcompact(io,r) : show(io,r)
+ compact ? showcompact(io,r) : rshow(io,r)
if signbit(i)==1 && !isnan(i)
i = -i
print(io, compact ? "-" : " - ")
else
print(io, compact ? "+" : " + ")
end
- compact ? showcompact(io, i) : show(io, i)
+ compact ? showcompact(io, i) : rshow(io, i)
if !(isa(i,Integer) || isa(i,Rational) ||
isa(i,Float) && isfinite(i))
print(io, "*")
View
4 base/dict.jl
@@ -16,9 +16,9 @@ function show(io, t::Associative)
for (k, v) = t
first || print(io, ',')
first = false
- show(io, k)
+ rshow(io, k)
print(io, "=>")
- show(io, v)
+ rshow(io, v)
end
print(io, "}")
end
View
4 base/expr.jl
@@ -40,13 +40,13 @@ isequal(x::Symbol , y::SymbolNode) = is(x,y.name)
function show(io, tv::TypeVar)
if !is(tv.lb, None)
- show(io, tv.lb)
+ rshow(io, tv.lb)
print(io, "<:")
end
print(io, tv.name)
if !is(tv.ub, Any)
print(io, "<:")
- show(io, tv.ub)
+ rshow(io, tv.ub)
end
end
View
2  base/io.jl
@@ -270,7 +270,7 @@ sprint(f::Function, args...) = sprint(0, f, args...)
function repr(x)
s = memio(0, false)
- show(s, x)
+ rshow(s, x)
takebuf_string(s)
end
View
2  base/rational.jl
@@ -31,7 +31,7 @@ function show(io, x::Rational)
if isinf(x)
print(io, x.num > 0 ? "Inf" : "-Inf")
else
- show(io, num(x)); print(io, "//"); show(io, den(x))
+ rshow(io, num(x)); print(io, "//"); rshow(io, den(x))
end
end
View
8 base/regex.jl
@@ -51,9 +51,9 @@ function show(io, re::Regex)
if (re.options & PCRE.EXTENDED ) != 0; print(io, 'x'); end
else
print(io, "Regex(")
- show(io, re.pattern)
+ rshow(io, re.pattern)
print(io, ',')
- show(io, re.options)
+ rshow(io, re.options)
print(io, ')')
end
end
@@ -70,12 +70,12 @@ end
function show(io, m::RegexMatch)
print(io, "RegexMatch(")
- show(io, m.match)
+ rshow(io, m.match)
if !isempty(m.captures)
print(io, ", ")
for i = 1:length(m.captures)
print(io, i, "=")
- show(io, m.captures[i])
+ rshow(io, m.captures[i])
if i < length(m.captures)
print(io, ", ")
end
View
2  base/set.jl
@@ -8,7 +8,7 @@ Set() = Set{Any}()
Set(x...) = Set{Any}(x...)
Set{T}(x::T...) = Set{T}(x...)
-show(io, s::Set) = (show(io, typeof(s)); show_comma_array(io, s,'(',')'))
+show(io, s::Set) = (rshow(io, typeof(s)); show_comma_array(io, s,'(',')'))
isempty(s::Set) = isempty(s.hash)
length(s::Set) = length(s.hash)
View
61 base/show.jl
@@ -3,13 +3,40 @@
show(x) = show(OUTPUT_STREAM::IOStream, x)
print(io::IOStream, s::Symbol) = ccall(:jl_print_symbol, Void, (Ptr{Void}, Any,), io, s)
-show(io, x) = ccall(:jl_show_any, Void, (Any, Any,), io::IOStream, x)
-showcompact(io, x) = show(io, x)
+function show(io, x)
+ if isa(io, IOStream) ccall(:jl_show_any, Void, (Any, Any,), io, x)
+ else default_show(io, x)
+ end
+end
+
+default_show(io::IO, x::Union(Type, Function)) = print(io, repr(x))
+function default_show(io::IO, x)
+ isa(typeof(x), CompositeKind) ? show_composite(io, x) : print(io, repr(x))
+end
+
+function show_composite(io, x)
+ T::CompositeKind = typeof(x)
+ names = filter(name->(name!=symbol("")), [T.names...])
+ values = {}
+ for name in names
+ try push(values, getfield(x, name))
+ catch err; error("default_show: Error accessing field \"$name\" in $T")
+ end
+ end
+ print(io, T.name, '('); show_comma_list(io, values...); print(io, ')')
+end
+show_comma_list(io::IO) = nothing
+function show_comma_list(io::IO, arg, args...)
+ rshow(io, arg)
+ for arg in args print(io, ", "); rshow(io, arg) end
+end
+
+showcompact(io, x) = rshow(io, x)
showcompact(x) = showcompact(OUTPUT_STREAM::IOStream, x)
show(io, s::Symbol) = print(io, s)
-show(io, tn::TypeName) = show(io, tn.name)
+show(io, tn::TypeName) = rshow(io, tn.name)
show(io, ::Nothing) = print(io, "nothing")
show(io, b::Bool) = print(io, b ? "true" : "false")
show(io, n::Integer) = (write(io, dec(n));nothing)
@@ -30,7 +57,7 @@ end
function show(io, l::LambdaStaticData)
print(io, "AST(")
- show(io, l.ast)
+ rshow(io, l.ast)
print(io, ")")
end
@@ -46,7 +73,7 @@ function show_delim_array(io, itr, op, delim, cl, delim_one)
if newline
if multiline; println(io); end
end
- show(io, x)
+ rshow(io, x)
if done(itr,state)
if delim_one && first
print(io, delim)
@@ -94,29 +121,29 @@ function show(io, e::Expr)
elseif is(hd,:return)
print(io, "return $(e.args[1])")
elseif is(hd,:string)
- show(io, e.args[1])
+ rshow(io, e.args[1])
elseif is(hd,symbol("::"))
- show(io, e.args[1])
+ rshow(io, e.args[1])
print(io, "::")
- show(io, e.args[2])
+ rshow(io, e.args[2])
elseif is(hd,:quote)
show_quoted_expr(io, e.args[1])
elseif is(hd,:body) || is(hd,:block)
println(io, "\nbegin")
for a in e.args
print(io, " ")
- show(io, a)
+ rshow(io, a)
println(io)
end
println(io, "end")
elseif is(hd,:comparison)
for a in e.args
- show(io, a)
+ rshow(io, a)
end
elseif is(hd,:(.))
- show(io, e.args[1])
+ rshow(io, e.args[1])
print(io, '.')
- show(io, e.args[2])
+ rshow(io, e.args[2])
else
print(io, hd)
show_comma_array(io, e.args,'(',')')
@@ -136,7 +163,7 @@ function show_quoted_expr(io, a1)
println(io, "\nquote")
for a in a1.args
print(io, " ")
- show(io, a)
+ rshow(io, a)
println(io, )
end
println(io, "end")
@@ -166,7 +193,7 @@ function show(io, e::TypeError)
end
end
-show(io, e::LoadError) = (show(io, e.error); print(io, "\nat $(e.file):$(e.line)"))
+show(io, e::LoadError) = (rshow(io, e.error); print(io, "\nat $(e.file):$(e.line)"))
show(io, e::SystemError) = print(io, "$(e.prefix): $(strerror(e.errnum))")
show(io, ::DivideByZeroError) = print(io, "error: integer divide by zero")
show(io, ::StackOverflowError) = print(io, "error: stack overflow")
@@ -186,7 +213,7 @@ function show(io, e::MethodError)
end
function show(io, bt::BackTrace)
- show(io, bt.e)
+ rshow(io, bt.e)
t = bt.trace
# we may not declare :_jl_eval_user_input
# directly so that we get a compile error
@@ -212,7 +239,7 @@ function show(io, m::Method)
if !isempty(tv)
show_delim_array(io, tv, '{', ',', '}', false)
end
- show(io, m.sig)
+ rshow(io, m.sig)
li = m.func.code
if li.line > 0
print(io, " at ", li.file, ":", li.line)
@@ -225,7 +252,7 @@ function show(io, mt::MethodTable)
d = mt.defs
while !is(d,())
print(io, name)
- show(io, d)
+ rshow(io, d)
d = d.next
if !is(d,())
println(io)
View
2  base/string.jl
@@ -1,6 +1,6 @@
## core text I/O ##
-print(io::IO, x) = show(io, x)
+print(io::IO, x) = rshow(io, x)
print(io::IO, xs...) = for x in xs print(io, x) end
println(io::IO, xs...) = print(io, xs..., '\n')
View
2  base/util.jl
@@ -84,7 +84,7 @@ function whicht(f, types)
while !is(d,())
if is(d.func.code, lsd)
print(stdout_stream, f.env.name)
- show(stdout_stream, d); println(stdout_stream)
+ rshow(stdout_stream, d); println(stdout_stream)
return
end
d = d.next
View
161 extras/recshow.jl
@@ -0,0 +1,161 @@
+# recshow.jl: show for self-referential and DAG structured objects
+
+# ---- ObjNode: capture of an object's output to show() -----------------------
+
+type ObjNode
+ # actual capture
+ obj
+ reused::Bool
+ items::Vector # ObjNode:s and strings/chars
+
+ # scratch space for recshow etc
+ strlen::Integer
+ name::String
+
+ ObjNode(obj) = new(obj, false, {}, -1, "")
+end
+emit(dest::ObjNode, arg) = (push(dest.items, arg); nothing)
+
+function print(io::IO, node::ObjNode)
+ if node.reused print(io, node.name)
+ else print(io, node.items...)
+ end
+end
+
+get_strlen(node::ObjNode) = node.strlen
+get_strlen(c::Char) = 1
+get_strlen(s::String) = strlen(s)
+get_strlen(x) = -1
+
+function finish!(node::ObjNode)
+ lengths = [get_strlen(item) for item in node.items]
+ if !any(lengths .== -1) node.strlen = sum(lengths) end
+end
+
+
+# ---- RecordIO: IO that recursively captures the output of show() ------------
+
+type RecordIO <: IO
+ shows::ObjectIdDict # Shows that have started capture so far
+ dest::ObjNode # Currently capturing
+end
+emit(io::RecordIO, arg) = emit(io.dest, arg)
+
+## Stuff needed to make an IO subtype: ##
+
+# Redirect character output to one place: emit
+print(io::RecordIO, s::ASCIIString) = emit(io, s)
+print(io::RecordIO, s::ASCIIString) = emit(io, s)
+print(io::RecordIO, s::UTF8String) = emit(io, s)
+print(io::RecordIO, s::RopeString) = emit(io, s)
+print(io::RecordIO, s::String) = emit(io, s)
+print(io::RecordIO, c::Char) = emit(io, c)
+
+write(io::RecordIO, c::Char) = emit(io, c)
+write(io::RecordIO, s::ASCIIString) = emit(io, s)
+
+# Work around some types that do funky stuff in show
+show(io::RecordIO, x::Float32) = print(io, repr(x))
+show(io::RecordIO, x::Float64) = print(io, repr(x))
+show(io::RecordIO, x::Symbol) = print(io, string(x))
+
+## Recording of show() ##
+
+type RecordShowError <: Exception
+ cause::Exception
+end
+function show(io::IO, e::RecordShowError)
+ println(io, "Exception in recshow:"); show(io, e.cause)
+end
+
+function record_show!(shows::ObjectIdDict, dest::ObjNode)
+ @assert !has(shows, dest.obj)
+ @assert isempty(dest.items)
+
+ shows[dest.obj] = dest
+ try
+ show(RecordIO(shows, dest), dest.obj)
+ catch e
+ if !isa(e, RecordShowError)
+ emit(dest, "#encountered exception!")
+ e = RecordShowError(e)
+ end
+ throw(e)
+ end
+ finish!(dest)
+ nothing
+end
+
+function rshow(io::RecordIO, arg)
+ if has(io.shows, arg) # reuse old node
+ node = io.shows[arg]
+ node.reused = true
+ emit(io, node)
+ else # record new node
+ node = ObjNode(arg)
+ emit(io, node)
+ record_show!(io.shows, node)
+ end
+end
+
+record_show!(dest::ObjNode) = record_show!(ObjectIdDict(), dest)
+record_show(arg) = (dest=ObjNode(arg); record_show!(dest); dest)
+
+
+# ---- list_trees!: Prepare recshow print list from ObjNode:s -----------------
+
+# is x immutable up to where show() calls rshow()?
+is_immutable_to_rshow(x::Union(Number,Function,Type,TypeName,Symbol)) = true
+is_immutable_to_rshow(x) = false
+
+function treeify!(trees::Vector{ObjNode}, node::ObjNode)
+ if !node.reused ||
+ (is_immutable_to_rshow(node.obj) && (0 <= node.strlen <= 11))
+ # node will be printed inline
+ node.reused = false
+ treeify_node!(trees, node)
+ else
+ if (node.name != "") return end
+ # First encounter: name the node, add it to the print list
+ push(trees, node)
+ k = length(trees)
+ node.name = "<x$k>"
+ end
+end
+treeify!(trees::Vector{ObjNode}, x) = nothing
+function treeify_node!(trees::Vector{ObjNode}, node::ObjNode)
+ for item in node.items; treeify!(trees, item); end
+end
+
+function list_trees!(args...)
+ trees = ObjNode[]
+ for arg in args; treeify!(trees, arg); end
+ k = 1
+ while k <= length(trees); treeify_node!(trees, trees[k]); k += 1; end
+ trees
+end
+
+
+# ---- recshow: Show a possibly self-referential object -----------------------
+
+recshow(io::RecordIO, arg) = rshow(io, arg)
+function recshow(io::IO, arg)
+ node = ObjNode(arg)
+ try
+ record_show!(node)
+ catch e
+ print_recshow(io, node)
+ throw(e)
+ end
+ print_recshow(io, node)
+end
+recshow(arg) = recshow(OUTPUT_STREAM, arg)
+
+function print_recshow(io::IO, node::ObjNode)
+ trees = list_trees!(node)
+ if isempty(trees); print(io, node); return; end
+
+ node.name = "<obj>"
+ if !is(trees[1], node); enqueue(trees, node); end
+ for node in trees; println(io, node.name, "\t= ", node.items...); end
+end
Something went wrong with that request. Please try again.