TODO

BUGS:

- ThreadScope DEADLOCKs occasionally, more often with --debug, why?

- X Window System error sometimes?

- background of some widgets on Windows are white when they are grey in Linux

- Make ^C work on Windows

- resizing the panes causes a grab lockup?
  Happened to Mikolaj on 7.11.2011, too.

- fix, rewrite or disable partial redrawing of the graphs pane
  that causes many of the graphical bugs reported below

- rewrite lots of drawing code to sidestep the fixed point precision
  problem of cairo

- scrolling to the right, we get some over-rendering at the boundary
  causing a thick line

- (probably the same as above)
  the gray vertical lines get sometimes randomly darker when scrolling
  by moving the scrollbar handle slowly to the right (probably caused
  by testing an x coordinate up to integral division by slice or without
  taking into account the hadj_value, so the line is drawn
  many times when scrolling)

- scrolling when event labels are on chops off some labels

- rendering bug when scrolling: we need to clip to the rectangle being
  drawn, otherwise we overwrite other parts of the window when there
  are long bars being drawn, which makes some events disappear.

- sideways scrolling leaves curve rendering artifacts (e.g., the thicker
  fragments of the flat line at the end of the Activity graph)

- sometimes 2 labels are written on top of each other even at max zoom,
  e.g. "thread 3 yielding" and "thread 3 is runnable"

- may be a feature: filling graphs with colours is from line 1 upwards,
  not line 0, so lines at level 0 seem under the filled area, not level with it

- a few levels of zoom in and then zoom out sometimes results in only
  a right fragment of the timeline shown, no indication in the scrollbar
  that scroll is possible, but scrolling indeed fixes the view

- ticking the trace boxes off sometimes shows a black rectangle in place
  of the switched off graph

- for small screens with many HECs, make it possible to zoom the canvas
vertically (perhaps via M-mouse wheel?)


OTHER:

- sort bookmark view by time?

- Delete key deletes bookmarks?

- hotkey to add a bookmark?

- 25%/50%/75% percentiles for pool size, see
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.6199
figure 15
WARNING: unlike mean, the median and other percentiles can't be computed
from percentiles of subnodes of the trees --- they need the whole data
for the interval in question at each level of the tree. This increases
the cost from O(n) to O(n * log n), where n is the total number of samples.
Additional log n factor, in comparison with mean, is probably inevitable
unless we put the data in an array, because otherwise we have to
sort the data for each interval to find the k-th element.
An extra problem is that to get accurate percentiles for splices
that do not match a single subtree node, we have to get the whole
data for the splice again, completely repeating the calculations.
Then the preprocessing via creating the tree would only be useful
if the tree stores the whole data at each tree level, already partitioned
and the data for each slice may be gathered cheaply (but a bit inaccurately,
see the use of SparkStats.rescale in the current code) by only looking
at a few nodes of the tree at a single level, instead of traversing
a very long list. There are better data structures than the spark tree
for quick lookup of sorted data, so let's remove the pool sizes from
the spark tree altogether and hack them separately (or use the better
structures for everything). Use the trick with calculating percentiles
from all raw data, but after quantizing it into a histogram,
to make it tractable; the trick is orthogonal to the change of data
representation to multi-level array aggregates, but with old data
representation it may be too slow

- resample the data (morally) uniformly, unless the sampling
is changed from GC time points to equal intervals
(note that with resampling, with enough extra inserted sample points,
the median approaches the mean, so calculating the median for pool size
does not make sense; however, percentiles still make sense --- they are
not just mean*(n/m), e.g., for y=5, all percentiles are 5,
while for y=x, from 0 to 10, they differ, regardless of sample density
and uniformity, except trivial cases)

- remove the grey halo in trivial cases of pool size,
like the line:  ___/----, but keep min/max for ___/\___

- make sure the user understands that the _areas_ are proportional
to the total number and curve points to rates and that the green area
in spark creation and the total area in spark conversion are equal;
perhaps tell so in the help tab or makr it in the timeline text summary tab

- the same aggregate style for the activity graph, to see min/max
(the green area does convey the total runtime, so perhaps mark min
just as a line, no grey shadow); or just redo it completely as the pool graphs

- test, in particular the quality of sampling, on the parfib suite;
ideally generate sampled events from accurate events and compare visually
and/or numerically

- either change scale together with zoom level (and keep a colored box
showing how much pictures space corresponds to how many sparks (should be
probably constant)) or (this one implemented currently:)
start with the best scale for the complete view
and the let the peaks be clipped, let the user manually readjust scale then

- limit the use of save/restore and similar crude hacks

- in zoom out, activity and spark graphs seem cut off at the ends,
which is mathematically correct, but one more slice or even pixel
on each end would make it look better, without compromising correctness
too much (just make sure the extra space does not grow with zoom level!);
the sparks and Activity are already rendered with (up to?) one extra slice
at the ends, but somehow this does not show (perhaps one is not enough)

- perhaps enlarge the main timeline canvas to the right to the nearest tick

- perhaps draw the labels vertically and reduce the size of the Y axis area

- click and drag the view (or the selected interval,
  perhaps shift-click or control-click when starting to drag))

- draw the detailed view in the background

- bookmarks
  - save
  - measure the time between two markers

- search for events in the event window

- indicate when one thread wakes up another thread, or a thread is migrated;
perhaps draw lines/arrows between the events?

- event list view
  - respond to page up / page down / arrows (FOCUS)
  - interact with bookmarks

- left pane for
  - bookmarks
  - list of traces
    - traces would be a tree-view, e.g.
      * HECs
        * 0
        * 1 etc.
      * Threads
        * 0
        * 1 etc.
      * RTS
        * live heap
      * ETW / Linux Performance counters
        * cache misses
        * stalls
        * etc.
     - some way to reorder the traces? dragging and dropping?
     - when rendering threads, we want some way to indicate the time
       between creation and death - colour grey?

- a better progress bar

- animate zoom level transitions:
ways to make the zoom in/out less confusing for users
(e.g. the sudden appearance of spikiness once thresholds)
animating the transitions would make it clearer
generate the bitmap of the higher resolution new view,
and animate it expanding to cover the new view
it'd be quick since it's just bitmap scaling
so the user can see the region in the old view
that is expanding out to become the new view

- let the user set the interval size/scale,
as an advanced option, _separately_ for each graph stack,
each HEC, each visible region at the current zoom level,
each selected region (if they are implemented)

- a button for vertical zoom (clip graphs if they don't fit at that zoom)
and/or select regions of time in the view and zoom only that region of display;

- an option to automatically change scale at zoom in to take
into account only the visible and smoothed part of the graphs

- label coloured areas with a mouseover, according to what they represent

- move the text entents stuff out of drawing ThreadRuns

- overlay ETW events

- integrate strace events in the event view on Linux

- colour threads differently

- thread names rather than numbers

- live heap graph

- summary analysis

- perhaps draw the graphs even if only the fully-accurate,
per-spark events are present in the logs (by transforming them
to the sampled format)

- and/or extrapolate data for large zoom levels with scarce samples
by slated lines, not just horizontal lines (which work great with
numerous enough samples, though)

- merge adjacent 0 samples:
if we have equal adjacent samples we just take the second/last
can be done when it's still a list, before making the tree;
simple linear pass, and lazy too;
does not make sense in per-spark events mode

- use the extra info about when the spark pool gets full and empty;
we know when the spark pool becomes empty because we can observe
that the threads evaluating sparks terminate; similarly, we know overflow
only occurs when the pool is full; but note the following:
"dcoutts: also noticed that we cannot currently reliably discover when the spark
queue is empty. I had thought that "the" spark thread terminating
implied that the queue is empty, however there can be multiple spark
threads and they can also terminate when there are other threads on that
capability's run queue (normal forkIO work takes precedence over
evaluating sparks)"

- consider making the graphs more accurate by drilling down the tree
to the base events at each slice borders:
they don't go down to the base events at each slice boundary
but only at the two viewport borders (hence the extra slices
at the ends dcoutts noticed).
The current state generally this results in smoothing the curve,
but a side-effect is that the graphs grow higher visually
(the max is higher) as the zoom level increases.

- or increase the accuracy by dividing increments not by the slice
length (implicitly), but the length of the sum of tree node spans
that cover the slice, similarly as in gnuplot graphs

- perhaps we should change the spark event sampling to emit events
every N sparks rather than at GC; but only if experiments (do more accurate
sampling and compare the general look of the graphs)
show that linear extrapolation of the GC data is not correct
(large spark transitions happen in the long periods between GC
and we don't know when exactly) (the 4K pool size guarantees that at least
with large visible absolute spark transition, the invisible transitions
can't be huge in proportion to the visible ones, so then linear extrapolation
is correct))

- use adaptive interval, depending on the sample density at the viewed fragment

- perhaps, depending on sample density, alternate between raw data,
min/max, percentiles; so the raw data line slowly explodes into a band,
a big smudge, like a string of beads, that gets even more detailed
and perhaps wider, when data density/uniformity allows it;
in other words: a thin line means we only guess that's where the data might be
a thicker one, with mix/max means we have some data,
but too irregular/scarce to say more, and full thickness line,
with percentiles means we have enough data or evenly distributed enough
to say all

- have *configurable* colors, like in Eden; where to save them?

- Eden TV has good visualisation for messages between processes & nodes,
(steal it, when we do more work on Cloud Haskell)

- show time corresponding to the eventlog buffer flushing,
after the event for this introduced

- verify correctness of the input eventlog before visualizing; define a method
of passing the error report from ghc-events to TS and displaying it; have
a grace period when that can be disabled via an option, until RTS and validation
stabilize

- perhaps the far left and far right bars on the histogram should be in red,
since extreme spark sizes are usually bad (or mark it in some other way)

- or colour stacked bars of the histogram according to strategy label

- determine which bars represent too small and too large sparks
based on the estimation of spark overhead divided by spark duration, etc.

- dynamic strategy labelling (lots of stuff has to be done elsewhere, first)

- mousing over or clicking a bar should tell "1.3s worth of sparks (28 total)
that took between 16--64ms to evaluate"; people want to know the number
of sparks as well as the total eval time, in particular kosmikus wants
to know this

- either show "Loading" when calculating spark duration histogram
for a selected range on the timeline graph, or store precomputed data
in the data structures for speedy drawing of the main timeline graphs,
however they are (re)implemented in the end (mipmap or tress or anything)

- manage histogram interval selection as gimp does: drag to create a selection,
then move the selection, etc.

- extend the selection stuff, e.g. keep bookmarks sorted and let you select multiple bookmarks which would then give you a range selection

- add a new info / stats tab giving numeric info about the current selection, e.g., the amount of cpu time used on each hec, mutator, gc, also sparks, basically all the things that ./prog +RTS --blah  can give you, but for any time period, not just whole run

- perhaps the current "reset zoom" button could change the x and y scale of timeline and histogram to fit the current region (which initially and most of the time later on is just the whole eventlog)?

- perhaps we need a floating "resize me" button to pop up on the graph whenever your current view ought to be resized , e.g. clipping at the top, or tallest bar taking up less than half the vertical space, or put that in red letters into the popups that we'll have at some point, or mark the upper or lower edge of the graph red (depending if it's clipped or too small) and blink the y-zoom reset button (to the right of the x-zoom reset button) in red, too

- select a region and display it in a separate tab
(e.g. next to the events tab)

- scroll around the graph image via a small zoomed out window
"The Information Mural: A Technique for Displaying
and Navigating Large Information Spaces"

- zoom histogram for granularity: increase the number of bars,
keeping the min/max unchanged; that requires log with base other than 2

- zoom histogram for detail: keep the number of bars the same,
but change the scale to only show sparks between a narrower min/max;
for this log 2 is enough, with a constant offset

- compare any of the graphs for the whole time and for the interval: either
by opening/detaching window/pane dedicated to the interval data
or by a permanent tab in the bottom pane with selectable traces
that only ever show the data from the selected interval
and when you zoom it, the interval changes in the upper pane
(but histogram should not be among those traces, because it does not scroll,
does not have the time X axis, and generally does not fit and is confusing)

- perhaps use the tab with traces to guide what is exported to PDF/PNG

- events in the info pane should be searchable, and possibly numbered

- selecting a range of events (via a shift-click) in the info pane should create a selection as might be done with the mouse.

- select and copy events from the info pane to the OS clipboard

- perhaps, on the X scale of timeline, show only offsets, to save space:
that is, only give the full time for the left point of the view and all other
times relative to that, e.g., 2.356 s, +1ms, +2ms, etc.

- and add a way to add a bookmark for a user-specified event, or a kind of events

- medium-term: let the user configure which instant events shown in the trace

- medium-term: user-defined visualisation frequency for the new trace (at least)
and any other tricks needed to visualize instant events well (show density
even if many events at almost the same spot and so the shades of green cue
is not possible) and fast (in particular, don't draw many lines at the same spot)
some random user feedback on the "lots of events" use case of traceEvents:
1. I don't need/use bookmarking (I can use the visual clues to jump), or even grep the event log
2. I find the visualization helpful, but disturbing, because it hides other information
3. it would be great to be able to visually distinguish different "messages"
4. it would be nice if drawing wouldn't slow down significantly even for a relatively small number of events (i.e., 100 is noticeable slowdown already)

- perhaps re-add and make optional the feature of bookmarks from trace events for the use case where one actually wants the bookmarks for jumping, etc.

- make TS useful for par-monad: generate and visualize user code / libs events, make the same kind of granularity histogram as for sparks, etc.

- make TS useful for concurrent code: forkIO concurrency events, properly implement the event merging for the cluster use case

- make TS useful for sequential code, in addition to showing the GC pattern

- make TS useful for the cloud haskell use cases

- info panel with the info that ghc +RTS -s shows, but for the selected interval;
how many sparks of each kind in that time, etc.

- perhaps add events for memory allocation and memory reclaimed through GC to make per-interval +RTS -s possible, at least mark them as feature requests for ghc-events on Trac; also add other events required by +RTS -s; the wiki page is http://trac.haskell.org/ThreadScope/wiki/RTSsummaryEvents

- add finite machines for the GC and other events (after new events are added andold extended) and use validation profiles to count some of the summary data that is calculated by hand right now

- another possibility to visualize sparks when there are not many of them would be to show an activity graph purely for spark evaluation, ie only counting those threads that are doing spark evaluation