Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[maps] no geometry column found when editing ES|QL layers #176227

Closed
nreese opened this issue Feb 5, 2024 · 1 comment · Fixed by #176231 or #176236
Closed

[maps] no geometry column found when editing ES|QL layers #176227

nreese opened this issue Feb 5, 2024 · 1 comment · Fixed by #176231 or #176236
Assignees
@nreese
Labels
bug Fixes for quality problems that affect the customer experience impact:critical This issue should be addressed immediately due to a critical level of impact on the product. Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas

Comments

@nreese
Copy link
Contributor

nreese commented Feb 5, 2024 •
edited

Regression caused by #174585. #174585 added drop_null_columns to ES|QL requests. Maps runs ES|QL statement with LIMIT 0 to fetch columns. drop_null_columns changed the shape of the response resulting in maps being unable to find columns.

Response from get columns before drop_null_columns

{
  "columns": [
    {
      "name": "geo.coordinates",
      "type": "geo_point"
    }
  ],
  "values": []
}

Response from get columns after drop_null_columns. Notice how columns array is now empty, resulting in "Elasticsearch ES|QL query does not have a geometry column" error.

{
  "all_columns": [
    {
      "name": "geo.coordinates",
      "type": "geo_point"
    }
  ],
  "columns": [],
  "values": []
}

Steps

  1. install sample web logs
  2. create new map, add ES|QL layer
  3. Edit query, changing limit from default of "10,000" to "1,000". Click "Run". Notice how results are no longer displayed and the editor has a warning
Screenshot 2024-02-05 at 7 52 36 AM
@nreese nreese added bug Fixes for quality problems that affect the customer experience Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas impact:critical This issue should be addressed immediately due to a critical level of impact on the product. labels Feb 5, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-presentation (Team:Presentation)

@nreese nreese changed the title [maps] no geometry column found when when editing ES|QL layers [maps] no geometry column found when editing ES|QL layers Feb 5, 2024
@nreese nreese self-assigned this Feb 5, 2024
@nreese nreese mentioned this issue Feb 5, 2024
Merged
@stratoula stratoula mentioned this issue Feb 5, 2024
Merged
stratoula added a commit that referenced this issue Feb 6, 2024
@stratoula @jughosta
Make the drop_null_columns configurable in ES|QL search strategies (#…
43d93ba 
…176236)

## Summary

Fixes #176227

As not everyone is using expressions and the `?drop_null_columns`
significantly changing the output, I am making this configurable. So for
the expressions users it will always be true but for anyone else it
would be false unless they specifically configure it.

This PR is fixing it but Nathan is going to follow up:

- maps are setting the drop_null_columns to true (as it makes the
response smaller, which is always a benefit)
- add some FTs to test the output

---------

Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
fkanout pushed a commit to fkanout/kibana that referenced this issue Feb 7, 2024
@stratoula @jughosta @fkanout
Make the drop_null_columns configurable in ES|QL search strategies (e…
fd1bf8f 
…lastic#176236)

## Summary

Fixes elastic#176227

As not everyone is using expressions and the `?drop_null_columns`
significantly changing the output, I am making this configurable. So for
the expressions users it will always be true but for anyone else it
would be false unless they specifically configure it.

This PR is fixing it but Nathan is going to follow up:

- maps are setting the drop_null_columns to true (as it makes the
response smaller, which is always a benefit)
- add some FTs to test the output

---------

Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
nreese added a commit that referenced this issue Feb 7, 2024
@nreese @kibanamachine
[maps] fix no geometry column found when editing ES|QL layers (#176231)
3e55ab5 
Resolves #176227

PR updates maps ES|QL code to handle response shape when
`drop_null_values` is used. PR adds an integration test that will fail
if the ES|QL response shape changes.

#### Test steps
1. install sample web logs
2. create new map, add `ES|QL` layer
3. Edit query, changing limit from default of "10,000" to "1,000". Click
"run". Verify results are displayed
4. Set query that returns no results like, "bytes > 100000000". Verify
layer displays no results.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
fkanout pushed a commit to fkanout/kibana that referenced this issue Feb 8, 2024
@nreese @kibanamachine @fkanout
[maps] fix no geometry column found when editing ES|QL layers (elasti…
45a2efe 
…c#176231)

Resolves elastic#176227

PR updates maps ES|QL code to handle response shape when
`drop_null_values` is used. PR adds an integration test that will fail
if the ES|QL response shape changes.

#### Test steps
1. install sample web logs
2. create new map, add `ES|QL` layer
3. Edit query, changing limit from default of "10,000" to "1,000". Click
"run". Verify results are displayed
4. Set query that returns no results like, "bytes > 100000000". Verify
layer displays no results.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
@stratoula @jughosta @CoenWarmer
Make the drop_null_columns configurable in ES|QL search strategies (e…
4aa852e 
…lastic#176236)

## Summary

Fixes elastic#176227

As not everyone is using expressions and the `?drop_null_columns`
significantly changing the output, I am making this configurable. So for
the expressions users it will always be true but for anyone else it
would be false unless they specifically configure it.

This PR is fixing it but Nathan is going to follow up:

- maps are setting the drop_null_columns to true (as it makes the
response smaller, which is always a benefit)
- add some FTs to test the output

---------

Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
@nreese @kibanamachine @CoenWarmer
[maps] fix no geometry column found when editing ES|QL layers (elasti…
a93bf98 
…c#176231)

Resolves elastic#176227

PR updates maps ES|QL code to handle response shape when
`drop_null_values` is used. PR adds an integration test that will fail
if the ES|QL response shape changes.

#### Test steps
1. install sample web logs
2. create new map, add `ES|QL` layer
3. Edit query, changing limit from default of "10,000" to "1,000". Click
"run". Verify results are displayed
4. Set query that returns no results like, "bytes > 100000000". Verify
layer displays no results.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
@stratoula @jughosta @CoenWarmer
Make the drop_null_columns configurable in ES|QL search strategies (e…
d6f57e0 
…lastic#176236)

## Summary

Fixes elastic#176227

As not everyone is using expressions and the `?drop_null_columns`
significantly changing the output, I am making this configurable. So for
the expressions users it will always be true but for anyone else it
would be false unless they specifically configure it.

This PR is fixing it but Nathan is going to follow up:

- maps are setting the drop_null_columns to true (as it makes the
response smaller, which is always a benefit)
- add some FTs to test the output

---------

Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
@nreese @kibanamachine @CoenWarmer
[maps] fix no geometry column found when editing ES|QL layers (elasti…
2918505 
…c#176231)

Resolves elastic#176227

PR updates maps ES|QL code to handle response shape when
`drop_null_values` is used. PR adds an integration test that will fail
if the ES|QL response shape changes.

#### Test steps
1. install sample web logs
2. create new map, add `ES|QL` layer
3. Edit query, changing limit from default of "10,000" to "1,000". Click
"run". Verify results are displayed
4. Set query that returns no results like, "bytes > 100000000". Verify
layer displays no results.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
fkanout pushed a commit to fkanout/kibana that referenced this issue Mar 4, 2024
@stratoula @jughosta @fkanout
Make the drop_null_columns configurable in ES|QL search strategies (e…
13c6d26 
…lastic#176236)

## Summary

Fixes elastic#176227

As not everyone is using expressions and the `?drop_null_columns`
significantly changing the output, I am making this configurable. So for
the expressions users it will always be true but for anyone else it
would be false unless they specifically configure it.

This PR is fixing it but Nathan is going to follow up:

- maps are setting the drop_null_columns to true (as it makes the
response smaller, which is always a benefit)
- add some FTs to test the output

---------

Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
fkanout pushed a commit to fkanout/kibana that referenced this issue Mar 4, 2024
@nreese @kibanamachine @fkanout
[maps] fix no geometry column found when editing ES|QL layers (elasti…
a799135 
…c#176231)

Resolves elastic#176227

PR updates maps ES|QL code to handle response shape when
`drop_null_values` is used. PR adds an integration test that will fail
if the ES|QL response shape changes.

#### Test steps
1. install sample web logs
2. create new map, add `ES|QL` layer
3. Edit query, changing limit from default of "10,000" to "1,000". Click
"run". Verify results are displayed
4. Set query that returns no results like, "bytes > 100000000". Verify
layer displays no results.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nreese nreese

Labels
bug Fixes for quality problems that affect the customer experience impact:critical This issue should be addressed immediately due to a critical level of impact on the product. Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas
Projects
None yet
Milestone
No milestone
2 participants
@nreese @elasticmachine