RFC: Design of Scrollable Widgets

[joshka]

Problem

There are a few widgets that need scrolling behavior. It would be wasteful to implement this on each widget rather than doing it properly once. There is existing scroll behavior in Table, Paragraph and List. There are a few issues / PRs related to scrolling currently in the queue

• Table: Add scrollbar to table widget #173
• Paragraph: Add the scroll bar to the Paragraph widget #139
• Paragraph: I would like to have the ability to get the offset of the text being drawn in Paragraph. #136
• Paragraph: Vertical alignment for Paragraphs? #135
• feat(paragraph): improve scrolling #7
• feat(table): allow page up/down in table #4
• feat(widgets): add offset() and offset_mut() for table and list state #12
• feat(list): add offset method for ListState #8
• feat(list): Add padding and truncation #150
• manipulating ListState/TableState offset fdehau/tui-rs#495
• POC: Viewport scrolling #119
• Split scrollable viewports #121
  Elsewhere:
• Add support to set text wrapping rhysd/tui-textarea#13 has some discussion on scroll behavior for tui-textarea (specifically this comment)
• Using tui-rs swanandx/bnomial-cli#1
• Smooth Scrolling Rigellute/spotify-tui#229

Current Behavior

Paragraph:

• does not impl StatefulWidget
• implements scrolling as an x,y offset (Paragraph::default().scroll((1,2)))
• renders vertical scroll, by ignoring lines before the scroll offset and after the render area
• renders horizontal scroll by chopping the left part of the line off
• has no scroll bar

Table:

• impl StatefulWidget with TableState: { selected, offset }
• implements scrolling as a single vertical offset to the row that will be at the top of the scroll view
• overrides the offset to ensure that the selected row is visible
• each row has a height field (longer cells are truncated)
• each row can have a bottom margin
• the table can have a header which is an always visible row
• has no scroll bar
• renders only complete rows

List

• impl StatefulWidget with Liststate: { selected, offset }
• implements scrolling as a single vertical offset to the item that will be at the top of the scroll view
• overrides the offset to ensure that the selected item is visible
• each item has a calculated height based on the text content
• has a PR for padding and truncation
• has a PR for implementing widgets as items (which can be variable height)
• has no scroll bar
• renders only complete rows (though there is a PR for truncation

Elsewhere:

tui_rs_tree_widget::Tree

• impl StatefulWidget with TreeState { selected, offset, opened }
• scrolling is a vertical offset to item that will be at top of scroll view
• overrides to ensure selected is visible
• calculates item heights based on text.height()
• only renders full items
• no scroll bar

tui-logger

• TuiWidgetInnerState has an offset
• Not 100% sure how it renders just taking a small look at the code, but seems to have a wrapped lines concept, so I'm assuming probably similar to the calculations above
• similar to above

tui-textarea

• does not implement stateful widget
• Implements scrolling via a viewport: https://github.com/rhysd/tui-textarea/blob/main/src/widget.rs
• I need to dig more into this one

tui-input

• just provides the scroll part, not the widget part
• handles scroll as just the horizontal position of some text to be displayed by whatever method is useful. (Example uses a paragraph and passes the value from visual_scroll to the paragraph.scroll() method)

I'm sure there are other things to look at. Please add them if you know of them.

Features of solution

Brainstorming ideas
TODO: refine these as must-have, should-have, could-have, won't have (MoSCoW)

1. Will be used by each ratatui widgets that currently scroll: Paragraph, List, Table
2. Can be used by those that don't currently: Calendar, Barchart, Chart, Sparkline?
3. Can be used by external widgets that need scroll behavior
4. Scrolling based on a line / row / item basis
5. Supports non-scrolled areas in a widget (e.g. table or list header / footer)
6. Supports scrolling a specified item into view (e.g. selected list item, top / bottom item)
7. Supports single line / row / item scrolling (possibly both line and item in the same widget)
8. Supports multiple scrolling (mouse scroll is often nice to scroll by some amount > 1)
9. Supports scrolling by partial / full page
10. Supports / guides truncation / partial rendering of items
11. Only supports the StatefulWidget approach (not Widget)
12. Should avoid having to implement the same logic on each widget (e.g. struct over trait when reasonable)
13. Scrollbar
14. Hide scrollbar (for helping clipboard copy work)
15. Height calculation
16. Visible items calculation
17. Querying scroll direction (useful in a list, where scrolling up / down should show the full new item)
18. Smooth scroll (scrolling to an item / page in steps based on tick rate)
19. Should not alter existing traits to implement
20. Should be able to implement existing widget scroll config on top of this (and mark calls as deprecated to guide users to new implementation)
21. Should be keyboard and mouse friendly
22. Horizontal and vertical scroll bars
23. Show scrollbar options should be { Always, Auto, Never }

Questions / things to look into / choices

1. Are there requirements / constraints that I've missed?
2. Should the scroll methods return a Result to indicate when they cannot scroll in the requested direction? Or should they just let the render method just fix the invalid scroll?
   a. if the scroll returns a result, we can beep / flash / take some action to load more results when a user tries to scroll past the end / beginning of a list of items.
3. ScrollState or ViewPort approach?
4. Should the behavior for scrolling by items be seperated from the behavior for scrolling by lines?
5. Struct vs Trait for scroll behavior
6. How much of the common behavior can we pull out of widgets and into the types that support scrolling implementation (e.g. scrolling specific items into view and truncation)?
7. Does this conflict with anything currently being worked on?
   a. Maybe the flexbox PR? wip(layout): flex layout - draft / poc / rfc #23
8. Are there any other examples of scrolling currently being implemented elsewhere worth looking at?
   a. libraries in other languages?
9. Should we be able to scroll tables horizontally by cells? This behavior might be useful to implement a carousel object
10. How would infinite scroll be handled?
11. I wonder if it's worth splitting out methods for helping with layout / item bounds into a scrollable trait widgets can implement to make it easy to calculate bounds / visibility of items.

Possible implementation approaches

1. ScrollableWidget as a Wrapper around other widgets
   Pros:
   • Simple
     Cons:
   • (covered in this comment)
2. Scrollable Widgets implement a Scrollable trait which contains methods for handling scroll position
   Pros:
   • ?
     Cons:
   • Every time the widget is constructed, we need to rese
3. ListState / TableState etc. implement a Scrollable trait, and have to handle each scroll method.
   Pros:
   • easy obvious interaction (list_state.scroll_down())
     Cons:
   • duplicate implementation in each state that handles scrolling
4. Add a ScrollState type with all the scrolling behavior and include this in ListState, TableState etc.
   Pros:
   • no duplication
   • can put methods for calculating visible area in this
   • can use the same scroll state for multiple widgets!
     Cons:
   • extra step for handling scroll methods (list_state.scroll_state.scroll_down()

And for scrollbar:

1. Implement scrollbar in each widget (too much duplication)
2. Implement a ScrollBar widget that each scrollable widget can use during render.
3. Implement the ScrollBar as part of the Block widget.

[lthoerner]

I'm not sure I quite understand the library well enough to meaningfully contribute to this feature, but I fully agree that there needs to be a more consistent scroll solution, so just wanted to +1 this.

[sophacles]

There were a couple pull requests (#119 and #121) that are worth mentioning here. They aren't exactly the same but they suggest an interesting approach.

Instead of putting scroll logic in to each widget, we could create a ScrollableWidget wrapper that:

1. Allow each widget to set a min_width and min_height, and for StatefulWidget, the ScollableWidgetState defines scroll_x and scroll_y.
2. When calling ScrollableWidget.render(area, original_buf) check if the area is big enough to hold the minimum dimensions. (if so, just render like normal)
3. If not, create a new buffer with the minimal dimensions, and pass that to Widget.render() along with a rect that is (0, 0, min_width, min_height)
4. After the widget renders, copy the area bound by (scroll_x, scroll_y, scroll_x + rect.width, scroll_y + rect.height) from the the new buffer to the original buffer.

When scrolling, update ScrollableWidgetState appropriately.

(The above would need to be fleshed out a bit more, with scrollbars, and so on, but I think those are doable with an approach like this).

I've never looked at textarea, but if they are using viewports to do scrolling, it's probably a similar idea.

[joshka]

@Eyesonjune18 wrote:

I'm not sure I quite understand the library well enough to meaningfully contribute to this feature, but I fully agree that there needs to be a more consistent scroll solution, so just wanted to +1 this.

No worries. I appreciate you taking a look regardless.

@sophacles wrote:

There were a couple pull requests (#119 and #121) that are worth mentioning here. They aren't exactly the same but they suggest an interesting approach.

I'll add these to the list and ping @gibbz00 for their input on this as it sounds like they probably given it some thought already.

Instead of putting scroll logic in to each widget, we could create a ScrollableWidget wrapper

I thought about that approach a little for similar reasons. I quickly rejected it as viable, mainly because it requires the contained widget to render the entire widget rather than just the visible portion. Giving it a bit more thought though, I think there's a few more problems with it:

1. As mentioned, the contained widget is not aware of the actual necessary area to render, so it has to render everything. This could be annoying for large lists, or tables with many rows, or paragraphs / text areas with lots of formatting.
2. The largest downside (I think) is that it would make scrolling by item/row rather than line difficult
3. It makes it difficult for a widget to render using the full amount of space it's given.
4. It makes differentiation of horizontal / vertical scroll difficult (I think - not 100% certain of this).
5. It requires a bit more effort to think through how to do layout on every widget (which dimensions do we need to support - { min, max, default, ... }
6. It makes rendering partial scroll areas still a responsibility of the widget. E.g. Table has a header, and could easily be extended to have a footer. This would mean that a scrollable Table has a ScrollableWidget, while a scrollable Paragraph is wrapped in one. The ergonomics of the difference would be weird.
7. It makes Paragraph wrapping weird - what is the buffer width in a ScrollableWidget?
8. It makes it difficult for the contained widget to interact with the scrolling (e.g. what if I want to show something like a scroll position at the bottom of the scroll area, not at the bottom of the full area?)

Given that most of the internal widgets have features that would need to interact with the scrolling in some way or other (item scrolling, total height calculation, partial item rendering, line wrapping, headers / footers, etc.), I think it's pretty clear that this shouldn't be a wrapper.

I'm going to add the following to the initial comment:

Possible implementation approaches

1. ScrollableWidget as a Wrapper around other widgets
   Pros:
   • Simple
     Cons: (covered in this comment)
2. Scrollable Widgets implement a Scrollable trait which contains methods for handling scroll position
   Pros:
   • ?
     Cons:
   • Every time the widget is constructed, we need to rese
3. ListState / TableState etc. implement a Scrollable trait, and have to handle each scroll method.
   Pros:
   • easy obvious interaction (list_state.scroll_down())
     Cons:
   • repeated implementation in each state that handles scrolling
4. Add a ScrollState type with all the scrolling behavior and include this in ListState, TableState etc.
   Pros:
   • no duplication
   • can put methods for calculating visible area in this
   • can use the same scroll state for multiple widgets!
     Cons:
   • extra step for handling scroll methods (list_state.scroll_state.scroll_down()

And for scrollbar:

1. Implement scrollbar in each widget (too much duplication)
2. Implement a ScrollBar widget that each scrollable widget can use during render.
3. Implement the ScrollBar as part of the Block widget.

[sophacles]

Re problems with the wrapper approach, generally I agree, although re point 1: I don't have a problem with drawing everything the user hands us, data management doesn't belong in drawing functionality - it should be done outside the widget IMHO - that's a different discussion though (see #164 )

Re trait vs ScrollState - these can be be combined using provided methods. We implement a ScrollState that holds relevant data and whatever mutators (etc) are needed. We also create a Scrollable trait that requires one method to be implemented (scrollable_state_mut(&mut self) -> ScrollState) and implement the rest of the trait functionality for the user.

For example something like: https://gist.github.com/sophacles/53490288c367f1300c08ad16d152bab2

This has the added benefit that if the widget author wants to implement some different behavor for scroll_y (or whatever) they can override it in their implementation.

[joshka]

For the comment on 1, one example that might help make it easier to think about the issue is how would you handle centered text in a scrollable widget? Alternatively, how would I handle having a list that has millions of items with only the current 40 or so showing being rendered? Or a very long document showing in a paragraph. I'd still want the scrollbar to be useful there. (I've been meaning to give #164 some more thought - in case you're concerned about screaming into the void on that ;))

The scrollable + scroll state approach makes sense. I wonder if AsRef could be worth implementing?

I've spent a little bit of time playing with what ScrollState might look like in a branch at main...joshka:ratatui:feat-scroll - very wip right now just there to understand barriers to implementation, and to spark ideas around missing behavior. (I also have some not yet finished work around pulling out the scrollbar as a widget)

[hasezoey]

personally i would like it if the final scroll implementation would allow for the possibility of "overscroll" (/ maybe also "underscroll"?), like some text editors allow.
"overscroll" as in "scroll beyond the last line", most commonly "one full page" or "half page", implemented by displaying empty lines / characters past (/before) the content.
(to my understanding #229 currently allows that, unless the element that implements it somehow does not, like clamping to the values in the table)

maybe this is solved via some kind of explicit padding or should it be handled separately?

[jdisanti]

Quoting the downsides of the wrapping widget approach:

The largest downside (I think) is that it would make scrolling by item/row rather than line difficult

I assume you mean in cases where the size of items are known since variable sized items would make this difficult for both item and line scrolling in all approaches.

An examination of the wrapping approach from an API standpoint

Originally, I was thinking the wrapping approach would be the most natural and could be salvaged, but after digging into it more closely, it definitely seems broken.

Examining the this approach from an API standpoint, I observe a few things:

A) Wrapping a widget in a scrolling widget is natural and feels right:

ScrollableWidget::default()
    .inner(
        SomeWidget::default()
            .props(...))
    )
    .render(area, buf, self.scroll_state);

B) But if SomeWidget is a stateful widget, it's not obvious how to get its state into place unless ScrollableWidget is special and doesn't implement StatefulWidget, but rather, has its own render method (or an alternate trait) that takes two states. Or, alternatively, there is some kind of way to bind state to a widget.

C) It's not so nice from a storage of state standpoint since each widget ends up needing a separate state associated with it:

struct MyUi {
    some_widget_state: SomeWidgetState,
    some_widget_scroll_state: ScrollState,
    another_widget_state: SomeWidgetState,
    another_widget_scroll_state: ScrollState,
}

D) State makes nested scrollbars impossible. Although, why someone would want that is a valid question. If the API is done right, it shouldn't be a necessary use-case, or if someone really needs it, they can drop down to using the scrollbar widget manually. What I'm thinking scrollbar inception would look like:

ScrollableWidget::default()
    .inner(
        ScrollableWidget::default()
            .inner(
                SomeWidget::default()
                    .props(...)
            )
    )
    // Assuming render takes two states as mentioned in B, how do you get state to `SomeWidget`?
    .render(area, buf, self.outer_scroll_state, self.inner_scroll_state);

I think the only way to salvage it would be to make binding state to a widget possible, as in SomeWidget::default().state(self.some_widget_state). Then the nesting problem goes away, but you're still left with the problem in C.

Rendering the inner widget remains a problem, but I think that can be solved by adding some kind of PartialRender trait that has a render function that takes both the destination area rect and a source area rect. If all the built-in widgets implement PartialRender, then implementing it for custom widgets should be pretty easy. In fact, the original Widget::render function could just defer to the partial render with the source and dest being the same.

[joshka]

I assume you mean in cases where the size of items are known since variable sized items would make this difficult for both item and line scrolling in all approaches.

Yes - variable size items is the main part of that downside, but even fixed size items means that when you scroll down by one item, you have to scroll down a certain number of lines. This means that the widget and scroll container must have some sort of communication about how to interpret scroll locations.

[hasezoey]

C) It's not so nice from a storage of state standpoint since each widget ends up needing a separate state associated with it:
Then the nesting problem goes away, but you're still left with the problem in C.

how about making the ScrollState struct generic, so that both could be stored in the same thing, like ScrollState<SomeWidgetState> (and implement something like Deref to transparently use the other state, or some getter method)?

---

aside from maybe needing communication to agree what a scroll should be (as @joshka noted), it would also likely be necessary that the inner struct needs to communicate what the actual size / max size is, so that there can actually be a scrollbar (otherwise how would you know how far the scrollbar is?)

[joshka]

I'd encourage writing some code to show this. No need for a full PR implementing it. Instead show what it looks like from the callers perspective, where they create a widget that supports scrolling and render it. This removes any ambiguity over which parts of the problem we're solving and which parts we're not.

[EdJoPaTo]

I think I have a good approach for the Scrollbar configuration:

Just like the user configures a Border for a given widget the user should be able to supply a Scrollbar:

const widget = SomeWidget::new()
  .border(Border::bordered())
  .scrollbar(Some(Scrollbar::new(ScrollbarOrientation::VerticalRight)));

The widget itself then knows about its border and things like headers (like the table has).
It can then calculate the scrollbar area. For the vertical scrollbar this should match the scrollable content height and the full widget width to draw the scrollbar on top of a border.

let scrollbar_area = Rect {
    // Inner height to be exactly as the content
    y: inner_border_area.y,
    height: inner_border_area.height,
    // Outer width to stay on the right border
    x: full_area.x,
    width: full_area.width,
};

The widget also knows every aspect of the ScrollbarState by itself:
The length of its full scrollable content (content_length), the viewport_length (scrollbar_area.height in this case), and the position from its own WidgetState offset.

Each widget knows precisely the area where the scrollable content is. The user shouldn't need to configure some margin when the widget supports scrollbars by itself. The widget also knows the scrollbar state. The widget does not know how exactly the scrollbar should look, but that's given by the user set-able scrollbar (like Border is set).

I implemented this approach in my Tree widget v0.18.0. I think this approach is useful for going forward.

Open question: The widget should have some control over the scrollbar being vertical or horizontal. For some widgets its either not useful or both possible and the widget needs to check which scrollbar is which. Currently, the Orientation is part of the Scrollbar definition. Maybe there should be two widgets, VerticalScrollbar and HorizontalScrollbar? Only the correct one would work with the method to set the scrollbar which is relevant for the given widget. When the widget supports both it can have both scrollbar methods.

[EdJoPaTo]

Another question that comes up: scroll the offset or the selection?

With mqttui (and therefore my Tree and Binary Data Widget) I went for scrolling the offset which is something that felt more natural coming from normal GUIs.
But they don't normally have a selection, so it's not an exact parallel.

Since then, I noticed other places which do have selections and how they behave. htop for example has a weird in between, it scrolls the offset when possible and keeps the selection relative to the offset. When scrolling the offset isn't possible anymore, the selection is moved.
The inventory in Borderlands 2 scrolls the selection and the offset follows. This approach is probably closer to keyboard usage and always ensures the selection is in view which can be a benefit.

Widgets on ratatui allow an optional selection. When there is no selection the widget should still be scrollable in my opinion (which isn't the case for some widgets currently). I think because of the optional selection the scrolling for offset is the most natural as the behavior does not change between a selection or not.
But even with that the selection could be forced to be within the view with the new offset after scroll. For mqttui I did not do that and instead went with moving the offset back to the selection when an interaction with the selection happened (like keyboard) (See #174 (comment))

Are there other thoughts on this?

[joshka]

Another question that comes up: scroll the offset or the selection?

Good points - I suspect that both should be available. It's reasonable to have mouse scroll that does the offset, but have keyboard shortcuts for selection (and perhaps they should be able to do the offset as well). In particular, a list item that is larger than the screen height requires both types of scroll to be able to be displayed in full.

[EdJoPaTo]

Scrollable widgets should include a common ScrollBehavior struct. This should be the same for all widgets in an application in order to create consistent behavior across widgets. This should be consistent across multiple renders and is probably a compile-time-constant or only changed by settings within the application. Idea of such a struct:

struct ScrollBehaviour {
  scroll_ends_on: LastItemOnEnd | LastItemOnStart=Overscroll | Ignore,
  selection_always_in_view: true | false,
  mousewheel_scrolls: Selection | Offset,
  keep_end_in_view_when_nothing_selected: true | false,
  
  show_horizontal_scrollbar: Always | OnlyWhenScrollable | Never,
  show_vertical_scrollbar: Always | OnlyWhenScrollable | Never,
  horizontal_scrollbar_location: Top | Bottom | Never,
  vertical_scrollbar_location: Left | Right | Never,
}

That way widgets can have similar behavior across many widgets. When this is not shared between multiple widgets (like it's currently the case) this will continue to have different implementations across different widgets.

With this information widgets themselves should be able to create a HorizontalScrollbar / VerticalScrollbar. They also know whether they are applicable for horizontal / vertical scrolling or not. The information currently in the ScrollbarState should be passed via argument, this struct just keeps duplicate information generated on every render or already present in each WidgetState and should be removed.

As this will have a default behavior I think this could even be achieved without a breaking change. The current widgets::Scrollbar could be deprecated. The new HorizontalScrollbar / VerticalScrollbar should not be listed below widgets as they are just parts and not a full widget.

[joshka]

Scrollable widgets should include a common ScrollBehavior struct

Much agreement that this should be moved into the widgets like block currently is.

The information currently in the ScrollbarState should be passed via argument

Yeah, ScrollbarState was my idea when the ScrollBar was originally implemented. On reflection, it was a mistake to make that instead of putting the properties on the Scrollbar.

The new HorizontalScrollbar / VerticalScrollbar should not be listed below widgets as they are just parts and not a full widget.

I don't mind keeping these available to people (as building blocks for other widgets)

[EdJoPaTo]

I don't mind keeping these available to people (as building blocks for other widgets)

My idea was not to completely remove these building parts. But they should not be grouped as "full widgets" like a List or Table.

[thscharler]

I would suggest using a container widget that manages the scrollbars and
can implement common behaviour.

It would use the following two traits, one for the widget and one for the state.

/// Trait for a widget that can scroll.
pub trait ScrolledWidget: StatefulWidget {
    /// Get the scrolling behaviour of the widget.
    ///
    /// The area is the area for the scroll widget minus any block set on the [Scrolled] widget.
    /// It doesn't account for the scroll-bars.
    fn need_scroll(&self, area: Rect, state: &mut Self::State) -> ScrollParam;
}

This trait is called before rendering the widget itself.

This allows the scrolling-container to calculate the exact layout.
At least tui_tree_widget needs information from the state to calculate a height,
so that's a parameter too.

The next trait is for the state:

pub trait HasScrolling {
    /// Maximum offset that is accessible with scrolling.
    ///
    /// This is shorter than the length of the content by whatever fills the last page.
    /// This is the base for the scrollbar content_length.
    fn max_v_offset(&self) -> usize;

    /// Maximum offset that is accessible with scrolling.
    ///
    /// This is shorter than the length of the content by whatever fills the last page.
    /// This is the base for the scrollbar content_length.
    fn max_h_offset(&self) -> usize;

    /// Vertical page-size at the current offset.
    fn v_page_len(&self) -> usize;

    /// Horizontal page-size at the current offset.
    fn h_page_len(&self) -> usize;

    /// Current vertical offset.
    fn v_offset(&self) -> usize;

    /// Current horizontal offset.
    fn h_offset(&self) -> usize;

    /// Change the vertical offset.
    ///
    /// Due to overscroll it's possible that this is an invalid offset for the widget.
    /// The widget must deal with this situation.
    fn set_v_offset(&mut self, offset: usize);

    /// Change the horizontal offset.
    ///
    /// Due to overscroll it's possible that this is an invalid offset for the widget.
    /// The widget must deal with this situation.
    fn set_h_offset(&mut self, offset: usize);
}

The scrolling widget imposes no unit on the usize values used in this trait.
Those could be items, lines or something else. The interpretation lies solely
with the widget.

But this means that the scrolling widget can't make any connection between
screen space and the values used in this interface.

Page_len

That's why it gets the current page_len via the trait.
With the page_len known any fractional scrolling is easy.

Scrollbar

The values used for the scrollbar are offset + max_offset.
Where max_offset is the maximum allowed offset that ensures that a full page
can be displayed. This should solve the complaints about overscrolling.

Both page_size and max_offset are values that can easily be calculated while
rendering, or are a rather small extra burden.

Letting the widget do all these calculations gives the widgets enough freedom
to do whatever they need to do.

Setting the offset

With the current offset, page_len and max_offset known all the wanted
behaviour can be done.

Drawback

The one drawback I saw, was that currently there is a strong link between
the selected item and scrolling. The table blocks scrolling, if the
selected item would go out of view.

This is a bit annoying and would require one more switch to turn this off.
Some function like scroll_selected_to_visible() would be nice. It
could be invoked when navigating with the keyboard.

[thscharler]

Example

I tried this, and here are the docs:

ScrolledWidget
HasScrolling
the widget: Scrolled

I wrote adapters for List, Table, Paragraph and just to see if it works
for tui_textarea and tui_tree_widget too.

widgets

If you want to run it there is examples/sample1.rs with the crate.

// I only tried this on windows with windows terminal and with alacritty
// So some things might not work ...
// Ctrl-Q works always :)

The crate contains more stuff, but it's late alpha/early beta now.

[thscharler]

This list was put up as requirements:

Features of solution

• Will be used by each ratatui widgets that currently scroll: Paragraph, List, Table

Yes

• Can be used by those that don't currently: Calendar, Barchart, Chart, Sparkline?

Not as they are.

But a viewport widget could implement these traits and render any
of these to a temporary buffer.

I haven't tried it, but buffer seems good enough for this.

• Can be used by external widgets that need scroll behavior

Yes

I tried

• tui_textarea
  The information necessary exists, but there is no public api for it.
  Would need only a small patch.
  The example uses the cursor api to do something close.

• tui-rs-tree-widget
  It would work nicely if implemented directly for the widget.
  And it has enough public apis that a wrapper can work too.

• Scrolling based on a line / row / item basis

Abstract notion of an offset. Can denote any of the above.
Could be made configurable for a widget, if it wants to switch its scrolling logic.

• Supports non-scrolled areas in a widget (e.g. table or list header / footer)

Yes. Rendering is the job of the widget. Scrolling only handles some offset.

• Supports scrolling a specified item into view (e.g. selected list item, top / bottom item)

No. This must be implemented by the widget.

• Supports single line / row / item scrolling (possibly both line and item in the same widget)

Yes, but not at the same time. A widget could have a switch that changes its behaviour though.

• Supports multiple scrolling (mouse scroll is often nice to scroll by some amount > 1)

Definitively.

• Supports scrolling by partial / full page

Yes

• Supports / guides truncation / partial rendering of items

Only if the widget supports it.

• Only supports the StatefulWidget approach (not Widget)

Needs some place to store state. But an adapter should always be possible. (See ParagraphExt).

• Should avoid having to implement the same logic on each widget (e.g. struct over trait when reasonable)

There could be some struct to hold the data, but I doubt all widgets that could
support scrolling will make use of it. That's rather an implementation detail of
a specific widget.

• Scrollbar

Yes, both. On each side if wanted.

• Hide scrollbar (for helping clipboard copy work)

pub enum ScrollbarPolicy {
    Always,
    #[default]
    AsNeeded,
    Never,
}

With AsNeeded the widget can control it too.

• Height calculation

That's up to the widget.

• Visible items calculation

That's up to the widget too.

• Querying scroll direction (useful in a list, where scrolling up / down should show the full new item)

Don't understand what's meant here?

• Smooth scroll (scrolling to an item / page in steps based on tick rate)

That's out of scope. There's no way enough infrastructure to accomplish that.

Could add something like:

fn ticked_scroll_to(tick_state: &mut TickState);

with

struct TickState {
  start_offset: usize,
  end_offset: usize,
  tick: usize,
  steps: usize
}

which is driven by some external timer.

• Should not alter existing traits to implement

Adds two traits. One for the widget and one for the widget-state.

• Should be able to implement existing widget scroll config on top of this (and mark calls as deprecated to guide users
  to new implementation)

The current behaviour can stay. Maybe needs a flag to switch off the 'scroll-selected-to-visible'
behaviour that exists.

• Should be keyboard and mouse friendly

Mouse interactions can be centralized on the Scrolled widget.
Keyboard navigation rather is the domain of a specific widget.

List/Table may share some behaviour, but something like tree-widget will hardly conform.

• Horizontal and vertical scroll bars

Yes

• Show scrollbar options should be { Always, Auto, Never }

Yes

Questions / things to look into / choices

• Are there requirements / constraints that I've missed?

Wrapping text should probably not scroll at all, or maybe it needs
some extra width parameter which controls the wrap, and then it
could scroll horizontally.

• Should the scroll methods return a Result to indicate when they cannot scroll
  in the requested direction? Or should they just let the render method just fix
  the invalid scroll?

Letting the widget correct invalid offsets should be fine.
It has to validate the state anyway when it's rendering its content.

• a. if the scroll returns a result, we can beep / flash / take some action
  to load more results when a user tries to scroll past the end / beginning of a list of items.

TODO: Not covered yet.

• ScrollState or ViewPort approach?

ViewPort seems only useful for a line/column based approach.
There could be a Viewport widget that supports scrolling.
That one would fit into this proposal just fine.

• Should the behavior for scrolling by items be seperated from the behavior for scrolling by lines?

If a widget wants this differentation it can.
This will probably need to different rendering paths though.

• Struct vs Trait for scroll behavior

Trait

• How much of the common behavior can we pull out of widgets and into the types
  that support scrolling implementation (e.g. scrolling specific items into view and truncation)?

I don't think you can for the whole set of widgets that could scroll.
But it should be possible for some subsets. List and Table are very similar
as they are implemented now. Maybe Paragraph and tui-image could share some?

• Does this conflict with anything currently being worked on?

Maybe? I don't have enough overview.

• a. Maybe the flexbox PR? wip(layout): flex layout - draft / poc / rfc wip(layout): flex layout - draft / poc / rfc #23

That's a new layout-engine, as I understand it?
It will probably need a width to work against.
So basically the same as with wrapping Paragraph,
either don't scroll horizontally or use some extra
width for its purposes.

• Are there any other examples of scrolling currently being implemented elsewhere worth looking at?
  a. libraries in other languages?

I know Java/Swing. It uses a viewport. It has always been difficult to use when you have
something different from an image.

• Should we be able to scroll tables horizontally by cells? This behavior might be useful to
  implement a carousel object

That's out of scope, I think.

• How would infinite scroll be handled?

What's meant with that one?

• I wonder if it's worth splitting out methods for helping with layout / item bounds into
  a scrollable trait widgets can implement to make it easy to calculate bounds / visibility of items.

[hasezoey]

How would infinite scroll be handled?
What's meant with that one?

to my knowledge infinite scroll is basically loading more content on-demand seamlessly without paging when reaching the end (or near the end), for example on a list. (also see the wikipedia page)

[thscharler]

How would infinite scroll be handled?
What's meant with that one?

to my knowledge infinite scroll is basically loading more content on-demand seamlessly without paging when reaching the end (or near the end), for example on a list. (also see the wikipedia page)

Ok, so that's the "let's see if the logfile got longer in the meantime" case.

This could be part of every run of event-handling or some timer based refresh or some external
event source.

Or, with infinite scrolling the trigger would be some threshold that is reached, after handling
any navigation events on the list.

This should probably update the data-model and render the list-item with the new data.
The current offset could stay the same, just the scrollbar has a new limit and draws differently now.

Seen this way it is probably the normal workload of the application, not something that must
be especially considered for a scroll widget?

[thscharler]

Update on

• Can be used by those that don't currently: Calendar, Barchart, Chart, Sparkline?

Looked into creating a Viewport that renders into a temp buffer.
This would work fine for the above mentioned.

[achristmascarl]

creating a Viewport that renders into a temp buffer.

I tried something like this for a SQL client (scrollable.rs), but a major constraint I ran into is the max area of the temporary Buffer, which maxes out at u16::MAX; // 65_535u16.

In practice, at least for a Table, it's quite easy to hit this limit (15 columns * 16 width per column = 240; 150 rows * 2 height per row for a little padding = 300; you're already at 72_000 which is > u16::MAX), at which point the Buffer will start clipping the content.

The only workaround I could think of was to make the caller responsible for offsetting / "windowing" the content it asked to be rendered, but that was quite messy (viewer beware; caller is data.rs) and seemed like it defeated the purpose of the viewport approach. I gave up and plan on using the built in TableState offset for vertical scrolling and some custom logic for horizontal scrolling (or maybe a horizontal-only version of the viewport approach).

I'm a beginner at Rust/ratatui though, so if there's a different viable workaround, would appreciate any tips/pointers 🙏

[joshka]

Yeah, fixing the area = u16 constraint is something that needs to happen someday. It's less about just making the change an more about working out why it's there in the first place (no one really knows the history of why it's there).

Re: long tables (and large widgets in general), the right approach is to not render stuff which is off screen rather than render everything to a buffer that is big.

[thscharler]

creating a Viewport that renders into a temp buffer.

I tried something like this for a SQL client (scrollable.rs), but a major constraint I ran into is the max area of the temporary Buffer, which maxes out at u16::MAX; // 65_535u16.

In practice, at least for a Table, it's quite easy to hit this limit (15 columns * 16 width per column = 240; 150 rows * 2 height per row for a little padding = 300; you're already at 72_000 which is > u16::MAX), at which point the Buffer will start clipping the content.

The only workaround I could think of was to make the caller responsible for offsetting / "windowing" the content it asked to be rendered, but that was quite messy (viewer beware; caller is data.rs) and seemed like it defeated the purpose of the viewport approach. I gave up and plan on using the built in TableState offset for vertical scrolling and some custom logic for horizontal scrolling (or maybe a horizontal-only version of the viewport approach).

I saw that quirks too, and using buffers that large just to throw most of it away
immediately is not a good use of cpu-time. When you start scrolling it will be noticeably laggy too. Using the offset is better.

If your dataset is big enough, just creating all the Row and Cell structs is noticeable. (On my computer it takes 3000x5 cells to reach render times of about 10ms).

I'm a beginner at Rust/ratatui though, so if there's a different viable workaround, would appreciate any tips/pointers 🙏

You might be interested in https://github.com/thscharler/rat-ftable. It's almost ready for beta, but I'm just working on the docs anymore. The rest should be quite stable. It has row-wise scrolling on the y-axis and char-wise scrolling on the x-axis, among other things.

[joshka]

You might be interested in https://github.com/thscharler/rat-ftable. It's almost ready for beta, but I'm just working on the docs anymore. The rest should be quite stable. It has row-wise scrolling on the y-axis and char-wise scrolling on the x-axis, among other things.

Oh, I don't think I put the name together until just now. That's a neat set of libs you've got going on there. Lots of neat ideas about how to do things. Keep at it :)

[thscharler]

You might be interested in https://github.com/thscharler/rat-ftable. It's almost ready for beta, but I'm just working on the docs anymore. The rest should be quite stable. It has row-wise scrolling on the y-axis and char-wise scrolling on the x-axis, among other things.

Oh, I don't think I put the name together until just now. That's a neat set of libs you've got going on there. Lots of neat ideas about how to do things. Keep at it :)

Thanks, and some kudos back to you. I took quite a few ideas from your various sketches floating around.

[lthoerner]

To my understanding, the reason Ratatui does this is the following:

Most terminals do not support more than 16-bit cursor indexes. The decision to use u16 to represent dimensions in the terminal was well-intentioned in that it is true to the functionality of the system it interacts with, but in the process, it introduces a few edge cases such as this one, particularly when any type of arithmetic is being applied to the underlying u16 values.

The "proper" way to solve this would be to only use u16 for any cursor position inputs in the API, and internally handle everything as usize. However, there are two problems with this. Firstly, many of these bugs may be introduced by the backend; I know that Crossterm in particular often uses u16 internally and externally, which could mean that simply fixing Ratatui's implementation may not fix the bugs that people are experiencing. Secondly, terminals are not standardized in any real sense, and I think that otherwise-unconventional numeric bounds like this should be enumerated in the documentation of a function and not built into the implementation itself. This is, of course, as long as the backends aren't truncating the value, which they very well might be (I'm too tired to investigate that right now).

[thscharler]

To my understanding, the reason Ratatui does this is the following:

Most terminals do not support more than 16-bit cursor indexes. The decision to use u16 to represent dimensions in the terminal was well-intentioned in that it is true to the functionality of the system it interacts with, but in the process, it introduces a few edge cases such as this one, particularly when any type of arithmetic is being applied to the underlying u16 values.

The "proper" way to solve this would be to only use u16 for any cursor position inputs in the API, and internally handle everything as usize. However, there are two problems with this. Firstly, many of these bugs may be introduced by the backend; I know that Crossterm in particular often uses u16 internally and externally, which could mean that simply fixing Ratatui's implementation may not fix the bugs that people are experiencing. Secondly, terminals are not standardized in any real sense, and I think that otherwise-unconventional numeric bounds like this should be enumerated in the documentation of a function and not built into the implementation itself. This is, of course, as long as the backends aren't truncating the value, which they very well might be (I'm too tired to investigate that right now).

Using a u16 for positions is not a noticeable limitation, for relative positions I use an i16, and it works fine.

The problem above is that Rect::area() calculates the area and limits that result to an u16 too. Which probably works for a Buffer for a terminal window, but is a real limit for a background buffer for scrolling purposes.