get_grid() doesn't work properly when adding/deleting rows (async delayed)

[gioxc88]

Below a simple reproducible example:

1. I have a function called get_add_delete_row_fn that adds 3 options to the right click context menu for deleting rows and adding rows above or below
2. The function works correctly on the front end (rows get deleted and added)
3. When I then call grid.get_grid() and display grid.grid_data_out['grid'] the output is not updated right away (it still shows the data previous to deletion/addition)
4. If I wait a few seconds and call grid.grid_data_out['grid'] the data is updated correctly.
5. Also if I rerender the grid, the changes are lost

this is the function to add the the context menu the option for deleting and inserting rows

import pandas as pd
import numpy as np
from ipyaggrid import Grid

def get_add_delete_row_fn(blank_row=None):
    blank_row = blank_row or 'null'
    fn = f'''
        function (params) {{
            let blankRow = {{}};
            params.columnApi.getAllColumns().forEach((column) => {{
              blankRow[column.getColId()] = '';
            }});

            if ({blank_row} !== null) {{ 
               blankRow = {blank_row};
            }};

            // console.log({blank_row});
            // console.log(blankRow);

            return [
              {{
                name: 'Delete Row(s)',
                action: function () {{
                  let selectedRows = params.api.getSelectedRows();
                  // console.log(selectedRows);
                  // let selectedData = selectedNodes.map(node => node.data);
                  params.api.applyTransaction({{ remove: selectedRows }});
                }},
              }},
              {{
                name: "Insert Row Above",
                action: function() {{
                  let newData = blankRow;
                  params.api.applyTransaction({{add: [blankRow], addIndex: params.node.rowIndex}});
                }},
              }},
              {{
                name: "Insert Row Below",
                action: function() {{
                  let newData = blankRow;
                  params.api.applyTransaction({{add: [blankRow], addIndex: params.node.rowIndex + 1}});
                }},
              }},
              'copy',
              'paste',
              'separator',
              // other context items...
            ];
          }}
    '''
    return fn

This is the grid creation

df = pd.DataFrame([{'a': i, 'b': i+1, 'c':i+2} for i in range(10)])
g = Grid(
    grid_data=df,
    grid_options={
        'columnDefs': [{'field': col} for col in df],
        'enableSorting': True,
        'enableFilter': True,
        'enableColResize': True,
        'enableRangeSelection': True,
        'getContextMenuItems': get_add_delete_row_fn(),
        'rowSelection': 'multiple'
    },
    theme='ag-theme-balham',
    show_toggle_edit=True,
    sync_on_edit=True,
)
g

Now to reproduce the example could you please:

1. Delete a few rows (select multiple rows by holding CTRL or SHIFT, then right click and Delete Row(s))
   (Tip: if you don't see the menu it's because it's covered by the jupyterlab menu, just scroll with the mouse wheel and it should appear)
2. Execute the following in a cell:

g.get_grid()
display(g.grid_data_out['grid'])

You will see that the data is not updated, but the cell is executed

4. Now please wait 1 or 2 seconds and execute only this g.grid_data_out['grid'] in another cell.
   Now the data is updated.
   

5. Finally try to rerender the grid in another cell and you'll see that the changes are lost
   

[aliallday]

I get a similar problem but for me it's not an issue of waiting 1 or 2 seconds. I think it's more related to displaying the grid again before accessing the underlying dataframe. If I update the data for the grid and then request g.grid_data_out['grid'], the grid is unchanged. I need to display the grid in a jupyter cell again after updating it. Then when I run g.grid_data_out['grid'] I see the updated results.

[gioxc88]

it would be great if this could be fixed as adding and deleting rows as a crucial part of many apps
I would to PR but unfortunately I think this is beyond my capabilities

[DiegoF90]

I have noticed that the grid gets updated if you run get_grid() in one cell and then use grid_data_out['grid'] to request the grid in the next one (does not seem to work if you do both in the same cell).

[gioxc88]

Yes this is the same issue!! hopefully they'll fix it :)

[gioxc88]

anything on this issue?

[mariobuikhuizen]

get_grid() sends a message to the front-end which then sends back the changed grid. This happens async, so the next line is executed before that. One could use grid.observe(update_fn, names='grid_data_out') and then call get_grid(), but that would not be triggered if the grid wasn't changed, and could unintentionally trigger the next time the grid is changed.

We could also sync the grid on the menu action, like what happens on the built-in edit:

g = Grid(
  ...,
  js_helpers_custom="""
        helpersCustom = {
            syncGrid() {
                view.model.set("_export_mode", "grid");
                view.model.trigger('change:_counter_update_data');
            }
        }
    """,

and call it from the menu items:

...
return [
    {{
      name: 'Delete Row(s)',
      action: function () {{
        ...
        params.api.applyTransaction({{ remove: selectedRows }});
        helpers.syncGrid();
      }},
    }},
...

Whole example:

import pandas as pd
import numpy as np
from ipyaggrid import Grid

df = pd.DataFrame([{'a': i, 'b': i+1, 'c':i+2} for i in range(10)])
g = Grid(
    grid_data=df,
    grid_options={
        'columnDefs': [{'field': col} for col in df],
        'enableSorting': True,
        'enableFilter': True,
        'enableColResize': True,
        'enableRangeSelection': True,
        'getContextMenuItems': """
         function (params) {
             return [{
                 name: 'Delete Row(s)',
                 action: function () {
                     let selectedRows = params.api.getSelectedRows();
                     params.api.applyTransaction({ remove: selectedRows });
                     helpers.syncGrid();
                 },
             }];
         }
        """,
        'rowSelection': 'multiple'
    },
    theme='ag-theme-balham',
    show_toggle_edit=True,
    sync_on_edit=True,
    js_helpers_custom="""
        console.log('view', view.model)
        helpersCustom = {
            syncGrid() {
                view.model.set("_export_mode", "grid");
                view.model.trigger('change:_counter_update_data');
            }
        }
    """,
)

g

Also, holding shift when right-clicking prevents the lan contect menu from showing.

Would this work for y'all?

[gioxc88]

get_grid() sends a message to the front-end which then sends back the changed grid. This happens async, so the next line is executed before that. One could use grid.observe(update_fn, names='grid_data_out') and then call get_grid(), but that would not be triggered if the grid wasn't changed, and could unintentionally trigger the next time the grid is changed.

We could also sync the grid on the menu action, like what happens on the built-in edit:

g = Grid(
  ...,
  js_helpers_custom="""
        helpersCustom = {
            syncGrid() {
                view.model.set("_export_mode", "grid");
                view.model.trigger('change:_counter_update_data');
            }
        }
    """,

and call it from the menu items:

...
return [
    {{
      name: 'Delete Row(s)',
      action: function () {{
        ...
        params.api.applyTransaction({{ remove: selectedRows }});
        helpers.syncGrid();
      }},
    }},
...

Whole example:

import pandas as pd
import numpy as np
from ipyaggrid import Grid

df = pd.DataFrame([{'a': i, 'b': i+1, 'c':i+2} for i in range(10)])
g = Grid(
    grid_data=df,
    grid_options={
        'columnDefs': [{'field': col} for col in df],
        'enableSorting': True,
        'enableFilter': True,
        'enableColResize': True,
        'enableRangeSelection': True,
        'getContextMenuItems': """
         function (params) {
             return [{
                 name: 'Delete Row(s)',
                 action: function () {
                     let selectedRows = params.api.getSelectedRows();
                     params.api.applyTransaction({ remove: selectedRows });
                     helpers.syncGrid();
                 },
             }];
         }
        """,
        'rowSelection': 'multiple'
    },
    theme='ag-theme-balham',
    show_toggle_edit=True,
    sync_on_edit=True,
    js_helpers_custom="""
        console.log('view', view.model)
        helpersCustom = {
            syncGrid() {
                view.model.set("_export_mode", "grid");
                view.model.trigger('change:_counter_update_data');
            }
        }
    """,
)

g

Also, holding shift when right-clicking prevents the lan contect menu from showing.

Would this work for y'all?

Hello @mariobuikhuizen I have tested it and works for me.
If I could make a suggestion I think it would be useful to have an example of how to add / delete rows in the doc and also maybe add the js function syncGrid amongst the js helpers already available to the user without having to define it in the custom helpers

Thanks a lot for this example

[gioxc88]

@mariobuikhuizen
Always related to the same problem that grid_data_out is updated async, the following:

If I update the grid using grid.update_grid_data() ,the frontend is update but grid.grid_data_out still is not in sync.
I think this mechanism needs to be changed as it is preventing the correct workflow of an app.
For example if you have:

• button1 that runs function1 for populating the grid
• button2 that runs a function2 which takes as inputs the grid data

This is a very common use-case which is not currently possible to achieve safely due to the async behaviour

Probably the best thing would be to have a parameter that governs the async/sync behaviour of the grid updates.
Is there anyway I can overcome this for now (having front end and backend grid data in sync)?

See below:

Right after creation it throws an error because grid.grid_data_out['grid'] doest exist yet:

After updating the grid data, the frontend is updated but grid.grid_data_out['grid'] is not:

I really hope we can find a solution for this problem
Thanks a lot @mariobuikhuizen

below the full code:

import pandas as pd
import numpy as np
from ipyaggrid import Grid

df = pd.DataFrame([{'a': i, 'b': i+1, 'c':i+2} for i in range(3)])
g = Grid(
    grid_data=df,
    grid_options={
        'columnDefs': [{'field': col} for col in df],           
        "domLayout": "autoHeight",
    },
    theme='ag-theme-balham',
    show_toggle_edit=True,
    sync_on_edit=True,
    columns_fit="auto",
    height=-1
)
g.get_grid()
display(g)
display(g.grid_data_out['grid'])

g.update_grid_data(pd.DataFrame().reindex_like(df).fillna(10))
g.get_grid()
display(g)
display(g.grid_data_out['grid'])

[gioxc88]

hello @mariobuikhuizen any update on this by any chance?
Thanks a lot