Suggested changes to the experience of dataframes being truncated

[IanButterworth]

DataFrame show() truncations are highly logical for safe general use, but there are some occasions where it would be nice to be able to override the default show() truncations.

For instance

function runanalysis() #produces a dataframe that has results that you want to display on screen and return as an object
   df = DataFrame(...) 
   return df
end

julia> results = runanalysis()
26×3 DataFrames.DataFrame #the report display is truncated as necessary

Currently the user has to do an additional, and unobvious show command to print the entire report.

julia> show(results, allrows = true, allcols = true)

I see two ways to improve this experience:

1. When DataFrames are truncated, provide a tip about show(results, allrows = true, allcols = true)
2. Allow temporary overriding of the truncations from within the function that wants to return the object

Even if the function has determined that no limit is acceptable for that function's output, 2 may be unsafe if it wasn't temporary, because you can't predict what the user might also try to print in their session.

Therefore, I propose:

DataFrames.forcenextshowall()

which only sustains the override for the next show function.

For instance, in DataFrames:

const forceshowall = Ref{Bool}(false)
function forcenextshowall()
    forceshowall[] = true
end

Defaults in the show functions could be changed to

allrows::Bool = forceshowall[] || !get(io, :limit, false),
allcols::Bool = forceshowall[] || !get(io, :limit, false),

and at the end of the show methods they would simply always set

forceshowall[] = false

to reset the override.

cc. @bkamins given we were discussing it in slack

[bkamins]

@nalimilan - as an additional comment. I think this is not needed in Jupyter Notebook, as there the user can set ENV["LINES"] and ENV["COLUMNS"] to whatever value the user likes.

For the REPL the problem is that it seems it is impossible to override the default behavior of stdout (and we cannot wrap it in IOContext as this is not always allowed).

I understand that @ianshmean's idea is that if a function returns a data frame it internally can set the flag he proposes that will be used only one time. The problem is that the user can do ; to avoid calling show and will get a strange result later.

When I was thinking about it an alternative (if we really wanted to allow it - as maybe other people will feel that show with kwargs is just enough) is the following:

1. in the future we will have metadata for data frames
2. we could then allow some printing-related metadata, e.g. the creator of the data frame could set some flag that this data frame should be always printed in full

[IanButterworth]

I hadn't thought about the ; override issue.. that is true.. Perhaps a timer could be set to revert the override after 5 seconds or something small

[nalimilan]

I get your line of reasoning, but I'm a bit worried that the implementation would be quite hacky. At any rate, the meta-data approach sounds cleaner than forcing the next show call to print everything.

I wonder what's the assumption here. Do we think the returned data frame has something special which would justify printing it in full all time, and if so, what? Or is it just that the function would like to ensure its result is printed? Why doesn't it print the result directly?

Finally, maybe that's a problem that should be tackled in the REPL itself? GUIs don't have this problem since they allow scrolling. Maybe there should be a way to tell the REPL that when printing a particular object the next time it should pass some IOContext properties?

[IanButterworth]

Why doesn't it print the result directly?

Because I want both the report to be printed in full and the dataframe object to be returned. If I print it within the function it will be shown twice given its printed truncated when returned.

I feel like the easiest fix here would be to print a tip about show(df allrows = true) if rows are truncated, and allcols for columns. Users just don’t seem to know that off hand.

[bkamins]

@ronisbr - when you add a section on printing in the manual can you please close this issue, as I hope such instruction should resolve this issue. We are not planning to add a global state of how printing should be done.