Skip to content

The SlickGrid Community

Jump to bottom Edit New page
Ger Hobbelt edited this page Feb 6, 2023 · 3 revisions

SlickGrid Community Activity

Before putting this repo together I trawled around a bit looking for activity on all the forked repos out there. I was surprised at the amount of activity - there were a number of people and groups of people who had done major work, often synchronising between each other's forks.

Some examples are:

  • SimpleGy who has been working on a 'pinned columns' implementation

    Other and related frozen row/column work has been done by

  • GerHobbelt who is a whopping 1000 or so commits ahead of the fork point. (link points at the bleeding edge there; this fork does not yet offer frozen rows and columns but does include row and column spanning (similar Excel's 'merge cells' feature), thanks to work by k0stya, ojotoxy, Celebio and others.

I think a lot of SlickGrid users would be interested in the bigger picture behind major active forks, so if you'd like to give an overview of what your fork does, and any goals you might have, please get in touch and I'll post it here.

About the features implemented or researched in those forks mentioned above

Pinned Columns (vs. Frozen Columns)

Alternative names for these two similar-but-not-alike features:

  • Sticky Columns
  • Grid Split

Of course, since the grid is 2D, the same goes for rows: Pinned Rows / Frozen Rows / Sticky Rows.

First off, let's consider the current Office spreadsheet products: Excel, LibreOffice Calc, etc.: those all have the Frozen Rows & Columns feature built-in (a.k.a. Grid Split): you click/select a cell in the grid and hit the Freeze/Split button or menu item and every cell to the left and top is now "frozen". I'll assume you have seen this happen as you used any one of those tools.

Pinned Columns is (at least to me [@GerHobbelt]) something different, but I've found not everyone uses it for the same idea: some use it as an alias for Frozen Columns as described above, where:

Frozen columns & Rows: the grid is ultimately split up in 4 quadrants, where only the cells in the right-bottom quadrant can move/scroll in any direction. The cells in the top-left quadrant don't move at all, no matter to which cell in the grid your focus currently travels. Top-right quadrant only scrolls in left-right direction and the bottom-left quadrant only scrolls up-down, while you travel across the grid.

Now, to me, Pinned Columns & Rows are more of the position: sticky persuasion and here is where it becomes hairy and opinions differ among designers.

Sticky (type A): you mark one or more arbitrary columns and/or rows as 'sticky' and the grid driver software should then try its utmost to ensure those columns and rows 'stick' within the visible viewport area, very similar to CSS position:sticky (demo). Generally, though, we want those marked rows & columns to stick to the left / top only, i.e. we're okay with them disappearing beyond the right / bottom edges of our viewport into the data grid.

That demo page about CSS position:sticky is nice as it also shows position:sticky;bottom:0 behaviour, which is where grid UX designers may have different opinions: generally I've found we want to have the stick-to-the-top and stick-to-the-left behaviour only. I do not know of any desktop spreadsheet, database or other grid-perusing application, which has this feature, but we wanted to have it (Sticky (type A)) for our advanced grid usage at visyond.com.

Sticky (type B): The additional stick-to-the-bottom / stick-to-the-right behaviour is arguably also part of the 'sticky' feature, but sometimes you want it, sometimes you don't. And no-body I've run into has specific, consistent, clearly identifying names for those two different types of 'sticky' behaviour. (top/left-stickiness-only or all-around-sticky?) Worse, I've observed myself using pinning, sticky and freezing 'interchangeably', which doesn't help when you are discussion three different behaviours. 😊

Local agreement has at least been that "pinning" should mean top/left-sticky-only, while "sticky" is then reserved for top/left/bottom/right-sticky-all-around behaviour.

When do you want this type of behaviour?

Well, for instance in large financial reporting and analysis sheets, pinning is much nicer to have than Excel & Company's freezing: complex models and reports often come, sensibly, with, ah 🤔, intermediate summary rows & columns: you don't just list items 1 to 300 and drop a summary line at the bottom, but you sub-categorize those items and, say, items 1-12 get their own sub-category summary line, after which we list items 13-and-so-on: with such a row/column data structure, it is very handy to have some sort of pinning available, because you might be very interested to keep an eye on all (or particular) sub-category summaries while you continue to the end and the grand total of it all.

And when the number of sub-categories, i.e. pinned/frozen rows/columns, grows too large, relative to the size of your viewport, you might want to consider additional means and actions to ensure your view remains optimally legible. ... 🤡 A pinning hierarchy perhaps, anyone? 🤦😵‍💫

And that's just one example of why you might want pinning rather than freezing.

Did anyone code it for SlickGrid?

My memory is hazy (writing this in 2023AD), but it might be that not only SimpleGy but k0stya or Celebio also had some attempt of pinning built in. I [GerHobbelt] certainly contemplated it as a feature that would also give us frozen row/columns -- you simply pin the first N rows and M columns and you've got yourself the 'frozen' grid behaviour right there! -- but I never implemented it in my open source fork. The team at www.visyond.com later did add a pinning feature to the grid but that was embargoed.

When you consider pinning (or sticky) in conjunction with merged cells and columns you will very quickly discover that implementing that combo of features is bloody hard; for one, you cannot do what I've seen everyone else out there do, which is chop the viewport into 4 quadrants and (more or less) manage them independently, but "in sync", so that the quadrants scroll in step with moving grid cell focus as you walk the grid (focus is always in only one of the quadrants, BTW). Hence you're stuck with managing the rows' and columns' position by hand in a single viewport and before the advent of CSS's position:sticky that required an effort bordering the criminally insane. And SlickGrid was selected by me, and used by our team, way before we could demand customers to only use browsers which fully support that position:sticky feature in both directions, without ruffling important feathers. (Important desks still more often than not had fancy machines with IE11 or similar gear and everyone considered that one of those facts of life. 😉)

The bottom line: if you want this, reckon with a journey, even today. It might be that @6pac now has something integrated, but even so many years later I've yet to see some grid management library out there offer a well-behaved combo of freezing/pinning/sticky-marking on the one hand, together with merged cells on the other hand: just consider what you need to do, UI/UX-wise, when a large, fat, merged cell travels through and your user either wants to edit it (F2?) while it's already partially obscured or decides to pin/freeze the left-most column of the merged cell: either way it's sticking out of the designated quadrant and you have some tough design and coding decisions to make... (And you can take it to the bank that the user-desired behaviour for this scenario is context dependent! 😜 )

Performance?

Oh, and another thing: when you decide to freeze, most implementations (not just these SlickGrid forks but almost everybody out there) do the 4-quadrants thing, which is then coded more or less 'naively' by, in essence, managing 4 partial viewports in sync. This means your grid management code is in reality managing 4(!) grids instead of just 1, even if the buggers share a grid data table copy and include virtualization, so your UI performance will decrease and thus UX will deteriorate. It then depends on the quality of the implementation and the complexity of the grid/cell render/layouts, if that performance decrease is noticeable by the user. It depends.

This (and anticipating some pretty fancy = costly cell layouts) was what drove me to consider sticking with a single viewport render (no pun intended), but I decided that would only be doable when the browsers had position:sticky support all around and the bugs ironed out of that new(ish) CSS feature.

Still a lot of work then, but you'ld at least have a decent chance. You'll have to decide what to do with those merged cells though and pray your technical/design solutions match user expectations sufficiently to make it a happy feature instead of a what-the-hell-did-it-just-do UI behaviour. 😅

Merged Cells

Ah, another feature that can be observed on all modern(ish) spreadsheet applications, e.g. Excel.

IIRC, k0stya was where I merged/stole that feature from. It was a nice approach, where merging was administered through an additional lookup table, which was queried whenever cells positions/state in the grid was requested by the SlickGrid virtual grid renderer: all you need to store then is the width/height (in cells) and the left/top-most column/row index of the merged cell: as you travel from top to bottom and left to right, you then know how many cells-to-skip when you hit a merged-cell entry in the lookup table. No need to keep any data for the regular, non-merged cells. Of course, then you run into a bit of hairiness when you jump-focus to a grid coordinate that's right smack in the middle of a merged cell: that part had a few bugs I recall, but I'll have to check the source code and the comments in there to refresh my memory how this went about exactly. Anyway, GerHobbelt had a working merged cells (row & column wise), which was tested often.

It might be that some of the other forks also had a merged cells feature; the bottom line here is that I ripped it off another fork and only had to see if I ran into behaviour I didn't like or bugs that needed fixing. I don't remember this to ever have been a hassle, so the implementation I took (from @k0stya?) was good quality.

Performance, performance, performance... and Grid Virtualization not good enough

Grid Virtualization, already introduced as a primary feature by M. Leibman when he created SlickGrid, simply means you only render the cells that you actually see in the viewport. Thus you do not load the browser's DOM with 'useless' HTML elements, you do not 'render' those invisible cells either, and thus you can expect a very nice, possibly optimal, grid scroll/traverse UX behaviour for arbitrary large/huge grids: iff your JavaScript performance is good enough (using the V8 engine or similar), then, by keeping the DOM size minimal, you'll be able to get the best render times possible for your grid.

Alas, I was anticipating grids which would contain cells which had more=fancier/complexer cell layouts than merely a DIV+SPAN+text(value), while testing on 'average machines' showed that SlickGrid performance was the best I could get, but still not enough, as sometimes the UX would be horrible: stuttering scroll, rickety stepping using keyboard control, etc.

Performance analysis showed it was still the SlickGrid renderer costing me and causing these slight hickups, so GerHobbelt has something extra built in: deferred & sliced rendering.

When I blame "the renderer", that was both DOM updates (JS code) and browser engine DOM render/repaint costs combined: you almost always have to 'interpret' those profiler results you see in the debug panel! You are always profiling a complex system, where multiple components interact and result in unanticipated performance results, both good and bad.

For a while I was amazed why another (open source) virtual grid/table library was noticeably faster than SlickGrid on the machines I tested it: analysis of that one turned up that they had combined the Virtual Grid (viewport render only) idea with HTML <TABLE>, i.e. they rendered their viewport DOM as a <TABLE> with <TR> rows and <TD> columns/cells, plus some smart-alecking to appropriately position those visible rows and cells within the <TABLE> element, so you'ld still get your properly sized scrollbars, just like SlickGrid. 🙇 ... I remained loyal to SlickGrid despite the lure.

Deferred & sliced rendering. Usually, you update the grid as you go (via keyboard or mouse/touch control) and let the browser cope with the DOM redraw/rendering once your event handler completes. The browser may be smart enough to group or otherwise bundle/defer multiple DOM updates' renders, but at the time that certainly wasn't enough -- while there are DOM/JS arguments for not implementing such a browser-grouped/deferred render update even today.

Today, with faster machines and an improved V8 engine almost anywhere, I expect the same problem to show... later. But it never goes away: if you want to quick-scroll through a huge grid, render updates always need to be handled carefully if you want to deliver smooth UX at all times.

Assuming those render updates cannot be interrupted by user input, which, AFAICT, still can't -- as that would imply entering event handlers while the DOM is in a possibly 'undetermined state' as you'ld be halfway through rendering/updating your DOM info and some datums would thus, by definition, be 'in limbo' (old or new, but unknown which of the two). Thus I argue the rendering must still be managed JS-side for top-notch UX in a web browser technology setting.)

So what does GerHobbelt do to manage/defer/minimize those renders? As you'll understand from the above, we not only need to manage the moments in time when the renderer will trigger and execute, but, in order to keep those, ah 🤔, 'unattentive intervals' as-short-as-possible, we also cannot update the virtual grid DOM like we used to do: the hickups in the UX are due to single renderer executions taking too long, making them noticeable to the user.

So I ripped out the existing renderer code in SlickGrid and moved it into a JS timer event. I also altered the code to update the visible part of the viewport DOM incrementally, i.e. in small(ish) chunks, instead of all at once.

The idea here is this: when updating the DOM of even the tiny(ish) grid viewport alone is too costly (Grid Virtualization has already thrown out and ignored any cell that we safely can ignore right now!), then we have to only update part of the viewport DOM for the browser renderer to repaint: we simply don't have time for more, apparently, anyway. Thus the choice to plonk it in a timer interrupt (JS event):

  • the timer will only fire iff the browser isn't too busy already with other important stuff (such as user input handling): setTimeout() already does that for you.
    • incidentally, back then I discovered setInterval() is not something you want to use for this sort of thing, as the intervals would accumulate and once one finally got fired, you'ld get a very swift bam!-bam!-bam!-bam! sequence of timer events all of a sudden, which is not what you're looking for here: you just want a little delay and a signal as often as possible that the coast is clear for some DOM updates, but you're not bound to the rigor of a clock here anyway.
  • when the timer event fires, you render only one or more rows of SlickGrid viewport, instead of the entire thing all at once.
    • here things get hairy again, and in two directions:
      1. you need to consider very carefully which behaviour you wish to exhibit towards the user: do you want to queue grid changes (=> DOM updates) or do you want to round-robin them? I chose the former, assuming the user is most focused on the latest cell visit/edit only and thus cares most about seeing those cell(s) update as quickly as possible, while older edits/visits may lag behind. Your application and desired UX may be different, so always reconsider this very carefully: my chosen approach here might be counter-productive for your scenarios.
      2. what shall your heuristics be to terminate the current DOM update? Do you simply render a single row and wait for the next timer event to be fired to discover the next row with 'dirty cells' and then update than one? That's very probably sub-par as that would multiply the total user-visible viewport update by the number of rows in the viewport (times to timer delay). So you'll have to come up with something smarter and test that on your users and yourself to see if it's satisfactory. (I don't recall what I did exactly -- I'll have to check the source code & comments for that -- but I believe I had a time measurement built in, which looked at the time spent since the last render/timer event, and if you observe no postponement there, plus you measure time spent on the current partial DOM update itself, you get an 'impression' of the performance of the given browser/machine at that moment, so you can 'guess' if it's safe to render another dirty grid row right now: if all has quieted down, you thus only spend one (or a few) timer events to update the entire viewport. Of course, this will be visible as partially cleared/broken grids in the viewport while the machine is busy processing your travel commands, but at least it then allows you to quickly scroll top-to-bottom and vice versa through a million-row table using keyboard control.

This was the most major change/improvement I made to SlickGrid, at least from my perspective then. Other major features (merged cells, resizable columns, ...) have been done by others and I merely merged them into my fork and tweaked or extended them and ironed out glitches.

The final state of affairs re the GerHobbelt fork is ho-hum hand-wavey as development of the open source branch petered out, while I was busy as CTO to get the visyond.com application grid going and some of the stuff done there would have been cool to feed back into the open source branch, we agreed that the young startup wouldn't benefit positively from doing this, where competitors could grab the hard-learned stuff and thus overtake our development and delivery effort at the time.

That concludes the story of the GerHobbelt fork to date; I am considering re-using SlickGrid for another application & purpose, but if you read this 2023AD or later:

  1. don't get your hopes up.
  2. grab the @6pac fork and take it from there: that one has seen consistent and continued maintenance and development while I was elsewhere living the 996 life of startups, not keeping my own public SlickGrid intact.

✋

  • @GerHobbelt

Navigation

SlickGrid Repo

Jump Start

Resources

Learning

Tests

Contact/Support

Clone this wiki locally