What is the server-side data in ui.aggrid really used for?

[NichtJens]

I am tinkering with a project where I have a dataframe that gets updated by user interactions and the current version should be displayed using an ui.aggrid.

I find myself struggling with the data representation in grid.options["rowData"].

Since this is a list, equivalent to the output of df.to_dict("records"), I need to convert between the "row ID" (what dataframes call "index") and an integer list index.

I do something like this (wrapped away in a function, of course) in many place:

row_index = next(
    i for i, d in enumerate(grid.options["rowData"])
    if d["index"] == row_id
)

I am also using run_grid_method("applyTransaction", ...), to sync to the client side without re-draw. This is also (implicitly) based around the row ID, except for insertion via add with an addIndex, but there's really no other way to deal with inserts, I guess.

If I am not mistaken, the representation equivalent to df.to_dict("index") would make my life/code far simpler.

So, I have two questions:

1. Why was df.to_dict("records") chosen?
2. What is grid.options["rowData"] really used for?

The first one is just because I am curious. I would prefer to switch to df.to_dict("index"), but this would break everyone's code, which is clearly a bad idea. Unless someone has a solution where the representation can be configurable?

The second is what could actually help me: I am anyway handling the server-side data myself. So, at the moment, I am keeping my own and ui.aggrid's server-side data in sync with the client-side data via applyTransaction. I am wondering if I could just stop doing that and only have my own representation synced via applyTransaction. Basically, I initialize from my server-side data and then apply deltas to the client.

So, the more concrete version of 2. would be:

3. Is there any drawback in letting grid.options["rowData"] go out of sync in such a scenario?

[falkoschindler]

Good questions! Let me answer them one by one.

1. Why was df.to_dict("records") chosen?

Because that's AG Grid's own data format, not really a NiceGUI design decision:
ui.aggrid is a thin wrapper, and the options dictionary is passed verbatim to AG Grid's createGrid() in the browser.
AG Grid defines rowData as an array of row objects, which is exactly what df.to_dict("records") produces.
A dict-of-dicts like df.to_dict("index") simply wouldn't be valid AG Grid options.
So making the representation configurable isn't really on the table -- the server-side dictionary mirrors the AG Grid options 1:1, and we'd like to keep it that way.

2. What is grid.options["rowData"] really used for?

NiceGUI itself never interprets it.
It's just part of the element's options property, which is serialized and sent to the browser whenever the element is rendered or updated:

• on initial page render,
• whenever the element is updated -- explicitly via grid.update(), but also implicitly whenever you call .classes(), .style(), .props() or register a handler with .on() after the grid has been rendered. Every update re-sends all props, and the client-side component then destroys and recreates the grid from options, including rowData.

The only place where NiceGUI writes to it is load_client_data(), which copies the client's current data back into options["rowData"].

3. Is there any drawback in letting grid.options["rowData"] go out of sync?

Yes, one big caveat: any of the updates listed above will silently reset the grid to the stale rowData, reverting all the deltas you applied via applyTransaction.
If you can guarantee that the element is never updated after its initial render, your approach works fine -- rowData then effectively degrades to "initial data".
But it's easy to break this accidentally, e.g. with a grid.classes(...) call somewhere down the line.

Since you keep your own ID-keyed representation anyway, the simplest robust pattern is probably to treat it as the single source of truth and regenerate the list from it after every change (or batch of changes):

grid.options["rowData"] = list(my_rows.values())

This is just an O(n) copy of references, so it's cheap even for frequent updates, covers insertions and deletions without any extra bookkeeping, and an unexpected re-render then shows correct data instead of stale data.
(Cell updates via my_rows[row_id][...] = ... even propagate without regenerating the list, because both structures share the same row dictionaries.)

Unrelated tip in case you don't use it yet: the getRowId grid option makes AG Grid use your "index" column as row ID:

grid = ui.aggrid({
    ...
    ':getRowId': '(params) => String(params.data.index)',
})

With this, applyTransaction updates/removals and run_row_method work directly with your IDs.
(Note that AG Grid requires row IDs to be strings.)

[NichtJens]

This is very useful. Thank you!

Regarding 1., the 1:1 mirror makes perfect sense, of course. It's a bit surprising that AG Grid stores the data as list-like, but the transactions, at least, seemingly operate more dict-like.

getRowId is a very good tip and I am using it already for this exact purpose. Before I had found it, I had to fiddle with row ID <=> indices even more ;)

---

Regarding the actual problem, I was thinking about exactly this:

grid.options["rowData"] = list(my_rows.values())

as well. But, in my case, it would look more like this:

grid.options["rowData"] = df.to_dict("records")

Which is not just a copy of references but a full copy of the data. I could just take this hit for simplicity. As long as the table is not huge, it should not be an issue. But if the table was actually huge, it probably would be better anyway to stream the data only per page or similar optimizations (which I don't consider in scope at the moment).

Alternatively, I might actually get simpler code by making it worse: I could keep three sets of data in sync: the dataframe, my_rows and grid.options["rowData"]. Applying deltas to my_rows would be far easier than to grid.options["rowData"]. And updating via the first line above would also be simple.

I can even see a third option: I could run load_client_data after each applyTransaction.