[Canvas] Filters function doesn't work with essql

[cqliu1]

Kibana version:
8.4.0+

Describe the bug:

Likely a side effect of #119044 and #132332. Expressions using the filters functions are not applying filters correctly to the essql datasource function. Many legacy workpads pre 8.1.0 use filters, but it looks like BWC for filters use hasn't been preserved which makes this a breaking change.

The issue is the essql function expects kibana_context type for the functions input instead of filter type which is the type that filters returns. The solution is to cast filters to kibana_context to restore BWC for usage of the filters function.

Steps to reproduce the issue with examples:

1. Add some test data

PUT testcanvas

POST testcanvas/_doc/1
{
"@timestamp": "2022-09-01T00:00:00Z",
"label": "1 sept"
}
POST testcanvas/_doc/2
{
"@timestamp": "2022-09-02T00:00:00Z",
"label": "2 sept"
}
POST testcanvas/_doc/3
{
"@timestamp": "2022-09-03T00:00:00Z",
"label": "3 sept"
}

2. Run the expression using filters:

filters

| essql query="SELECT * FROM testcanvas"

| table

| render

The results are not correct.

But when running the updated expression using kibana | selectFilter :

kibana

| selectFilter

| essql query="SELECT * FROM testcanvas"

| table

| render

[elasticmachine]

Pinging @elastic/kibana-presentation (Team:Presentation)

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServicesSv)

[dokmic]

@elastic/kibana-presentation This proposal doesn’t seem like the best option due to a few facts:

• Implementing the typecast will require a lot of effort on the data plugin side as we will have to implement adapters per each filter type. And all of that will be removed as soon as we remove the function.
• The filter function should be automatically migrated to the kibana_context function in the existing workpads, and it can only be used if a user types that explicitly.
• The essql function should have been updated anyway, as described in [Canvas] Adding support of kibana_context to datasource expressions. #121859.

Therefore, there are a few options:

• We can prioritize the filters function removal and upgrade the rest of the functions to accept kibana_context on input.
• We can update the filters function to return the kibana_context. And for the existing functions, there is already a cast to the filters type. The filters should be marked as deprecated so the users can see that in the editor.

WDYT? Either way, the change should be done on the Canvas side. Would you mind taking this over, as it is irrelevant to the expressions plugin?

cc @cqliu1

[cqliu1]

Since there was a migration added as a part of #119044, any legacy workpad from pre Kibana 8.1.0 using filters have been rewritten to kibana | selectFilter on import/upgrade, so the only time an element would use the filters function post Kibana 8.1.0 would be if a user wrote it in from muscle memory when manually editing their expression.

What we should do is have filters throw an error and persuade the user to use kibana | selectFilter instead so they're aware that filtering may be broken when you using the filters function.