[data grid] Improve Server-side support with new data source

[cherniavskii]

Summary 💡

The server-side data source will be implemented in a step-by-step manner, the issues it will be covered in are mentioned in the list below.

• A TypeScript interface with clear requirements of what the user has to provide for the data grid to have the necessary data. (Handled in [DataGridPro] Add data source interface and basic documentation #10543)
  • Start with a Collection interface where you provide the interface for the data collection that could be used by Toolpad in their server-side function. The goal is to have the configuration of the data source in a single place where the users can translate the pagination, filter, sorting models, etc. to whatever you have on the server (HTTP request, DB query, or something else)
  • It should handle the fetching first, and we can extend the interface with updating functions later.
• 🚧 [data grid] Implement server-side data source in tree data #3377
• [data grid] Implement server-side data source in row lazy loading #10857
• [data grid] Implement server-side data source in row infinite loading #10858
• [data grid] Implement server-side data source in row grouping #10859
• [data grid] Implement server-side data source with aggregation #10860
• [data grid] Implement server-side data source with pivoting #10861

Motivation 🔦

To make the integration of the data grid with users' data easier. Today we have https://mui.com/x/react-data-grid/row-updates/#lazy-loading but it could be a lot better.

Examples 🌈

• https://marmelab.com/react-admin/DataProviders.html
• @Janpot has an idea for the createCollection() function in the Toolpad

User requests

• https://mui.zendesk.com/agent/tickets/15724

[cherniavskii]

cc @mui/toolpad

[cherniavskii]

We have recently explored supporting lazy loading for tree data and row grouping, and this seems to be extending the original description of the data source.

With the data source, we want to support at least these use cases:

• Row grouping: lazy-load children rows.
  Currently, row grouping only works on the client side. This means that all the rows have to be fetched in advance. This is not scalable for large datasets.
• Tree data: lazy-load children rows.
  Same as with row grouping, tree data only works on the client side. We have a demo allowing lazy load children rows, but it's not a part of the core package. We need to support this natively.
• Hooking up server-side pagination, sorting, and filtering
  While it's possible to use server-side pagination, sorting, and filtering, each of these features has to be handled separately. We need to provide a way to hook up all of these features at once. It will also be beneficial for us if we have a centralized way of fetching data from the server (when not using client-side features).
  This effectively replaces paginationMode="server", sortingMode="server" and filterMode="server".

It’s preferable for the data source to be optional in the use cases that are currently supported by the data grid (like server filtering, sorting, pagination, etc.).

Benchmarks:

• https://www.ag-grid.com/javascript-data-grid/server-side-model/
• https://js.devexpress.com/Documentation/ApiReference/Data_Layer/CustomStore/
• https://www.telerik.com/kendo-react-ui/components/grid/data-operations/odata-server-operations/
• https://blazor.syncfusion.com/demos/datagrid/remote-data?theme=bootstrap4
• https://www.infragistics.com/products/ignite-ui-angular/angular/components/grid/remote-data-operations

[Janpot]

Just for cross-reference: We now have an implementation in Toolpad which supports pagination, filtering, sorting and all CRUD operations.

[avioralm]

@cherniavskii Any update on this development ?

[BrentFarese]

Seems like this is a priority for V7? Does the MUI team have any guidance on when this will land/be usable? I ask b/c our team is considering server-side pagination, sorting, filtering in the upcoming months and we'll have to roll our own solution or use this solution if it becomes available.

We are Premium users. Thanks!

[MBilalShafi]

Hey @BrentFarese, thank you for reaching out.
The first implementation of the server-side data source is already in progress and will include server-side pagination, sorting, filtering support, tree-data lazy-loading, and other related improvements.
I can't say how soon this will be available, but I can assure you it's one of the top priority areas for the team and hopefully, it will soon be available.

Meanwhile, may I know what specific features you currently use or plan to use in the future, do you also expect any advanced use cases like aggregation, row-grouping, etc. to be utilized in your project with server-side support?

CC @avioralm

[BrentFarese]

Hey @BrentFarese, thank you for reaching out. The first implementation of the server-side data source is already in progress and will include server-side pagination, sorting, filtering support, tree-data lazy-loading, and other related improvements. I can't say how soon this will be available, but I can assure you it's one of the top priority areas for the team and hopefully, it will soon be available.

Meanwhile, may I know what specific features you currently use or plan to use in the future, do you also expect any advanced use cases like aggregation, row-grouping, etc. to be utilized in your project with server-side support?

CC @avioralm

@MBilalShafi we don't plan to use advanced features for the initial implementation, just pagination, sorting, filtering support. I know timelines shift all the time, but is this work expected to land in 1-2 months or is it more like 4-6 months? Just gauging whether we wait for it or roll our own. Thanks!

[MBilalShafi]

is this work expected to land in 1-2 months or is it more like 4-6 months

1-2 months is the more likely period for the first implementation (including server-side filtering, sorting, and pagination), however, I'm curious to know if meanwhile there's a barrier for you to keep using the server-side features already supported by the Grid.

The following features are already available in the Grid:

• Server side sorting
• Server side filtering
• Server side pagination

[BrentFarese]

is this work expected to land in 1-2 months or is it more like 4-6 months

1-2 months is the more likely period for the first implementation (including server-side filtering, sorting, and pagination), however, I'm curious to know if meanwhile there's a barrier for you to keep using the server-side features already supported by the Grid.

The following features are already available in the Grid:

• Server side sorting
• Server side filtering
• Server side pagination

No there is not a barrier to us using the existing server-side features. We might have to do that depending on when we implement server-side logic and when MUI releases the initial version. Would like to avoid having to migrate tho so we'll follow releases closely. Thanks!

[avioralm]

@MBilalShafi We are using the pro plan in our company and it is getting harder and harder to implement new tabular data features without the data source.
Can you share some time estimation for this feature?

[MBilalShafi]

Hi @avioralm, thank you for reaching out.

I'd be interested to know more about specific features / use-cases you want to solve with the data source.

The first implementation of data source is expected to land in Q2 2024, however it will cover a subset of features with other advanced ones to follow up.

Here's the list of Server-side features targeted in first implementation of the data source.

• Filtering
• Sorting
• Pagination
• Loading plain data with data source
• Tree data lazy loading

Does that cover your use case?

[avioralm]

@MBilalShafi
Yes it is fully covers my needs
especially the data tree implementation