Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Geopoint type is not supported in JDBC and raw formats #313

Open
chloe-zh opened this issue Dec 9, 2019 · 2 comments
Open

Geopoint type is not supported in JDBC and raw formats #313

chloe-zh opened this issue Dec 9, 2019 · 2 comments
Labels
enhancement New feature or request SQL

Comments

@chloe-zh
Copy link
Member

chloe-zh commented Dec 9, 2019

The response from the ES supports the geopoint type, and the csv format suppports geopoint as well, as shows below.
Query:

`select geoip.location from kibana_sample_data_ecommerce limit 3`

Response:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4675,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "kibana_sample_data_ecommerce",
        "_type" : "_doc",
        "_id" : "8qqPsG4BSRXzyKXU0903",
        "_score" : 1.0,
        "_source" : {
          "geoip" : {
            "location" : {
              "lon" : 31.3,
              "lat" : 30.1
            }
          }
        }
      },
      {
        "_index" : "kibana_sample_data_ecommerce",
        "_type" : "_doc",
        "_id" : "86qPsG4BSRXzyKXU0903",
        "_score" : 1.0,
        "_source" : {
          "geoip" : {
            "location" : {
              "lon" : 55.3,
              "lat" : 25.3
            }
          }
        }
      },
      {
        "_index" : "kibana_sample_data_ecommerce",
        "_type" : "_doc",
        "_id" : "9KqPsG4BSRXzyKXU0903",
        "_score" : 1.0,
        "_source" : {
          "geoip" : {
            "location" : {
              "lon" : -118.2,
              "lat" : 34.1
            }
          }
        }
      }
    ]
  }
}

Request:

POST _opendistro/_sql?format=csv
{
  "query": "select geoip.location from kibana_sample_data_ecommerce limit 3"
}

Output:

geoip.location,geoip
,"{location={lon=31.3, lat=30.1}}"
,"{location={lon=55.3, lat=25.3}}"
,"{location={lon=-118.2, lat=34.1}}"

However, the JDBC and raw formats are not working fot the geopoint type.
Request:

POST _opendistro/_sql?format=csv // or format=jdbc
{
  "query": "select geoip.location from kibana_sample_data_ecommerce limit 3"
}

Output:

{
  "error": {
    "reason": "There was internal problem at backend",
    "details": "GEO_POINT fieldName types are currently not supported.",
    "type": "IllegalArgumentException"
  },
  "status": 500
}
@rupal-bq
Copy link
Contributor

rupal-bq commented Jul 8, 2020

While trying to connect Microsoft Excel with Open Distro for Elasticsearch ODBC Driver, information about all columns is retrieved with the query DESCRIBE TABLES LIKE kibana_sample_data_flights. This return all columns including geo-point type which are OriginLocation & DestLocation.

So to get data, Microsoft Excel is sending a query

select `FlightNum`, `Origin`, `OriginLocation`, `DestLocation`, `FlightDelay`, `DistanceMiles`, `FlightTimeMin`, `OriginWeather`,  `dayOfWeek`, `AvgTicketPrice`, `Carrier`, `FlightDelayMin`, `OriginRegion`, `DestAirportID`, `FlightDelayType`, `timestamp`, `Dest`, `FlightTimeHour`, `Cancelled`, `DistanceKilometers`, `OriginCityName`, `DestWeather`, `OriginCountry`, `DestCountry`, `DestRegion`, `DestCityName`, `OriginAirportID` from `kibana_sample_data_flights`

This fails with the error

{
  "error": {
    "reason": "There was internal problem at backend",
    "details": "GEO_POINT fieldName types are currently not supported.",
    "type": "IllegalArgumentException"
  },
  "status": 500
}

The ODBC driver uses format=jdbc while sending the request.

@dai-chen dai-chen added the SQL label Sep 11, 2020
@jordanw-bq
Copy link
Contributor

When running the ODBC driver tests with coverage, tests in itodbc_catalog fail with Segmentation fault: 11

$ make ccov-all
[ 60%] Built target aws-cpp-sdk-core
[ 84%] Built target odfesqlodbc
[ 87%] Built target gtest
[ 88%] Built target gtest_main
[ 89%] Built target ut_helper
[ 90%] Built target itodbc_helper
[ 91%] Built target itodbc_pagination
[ 91%] Built target ccov-libs
[ 91%] Built target ccov-clean
[ 91%] Built target ccov-preprocessing
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from TestPagination
[ RUN      ] TestPagination.EnablePagination
[       OK ] TestPagination.EnablePagination (672 ms)
[ RUN      ] TestPagination.DisablePagination
[       OK ] TestPagination.DisablePagination (51 ms)
[----------] 2 tests from TestPagination (723 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (723 ms total)
[  PASSED  ] 2 tests.

All tests passed
[ 91%] Built target ccov-run-itodbc_pagination
Scanning dependencies of target itodbc_catalog
[ 91%] Building CXX object IntegrationTests/ITODBCCatalog/CMakeFiles/itodbc_catalog.dir/test_odbc_catalog.cpp.o
[ 92%] Linking CXX executable /Users/jordanw/Development/OSSRepos/ODFESQL/sql-odbc/build/odbc/bin/itodbc_catalog
[ 92%] Built target itodbc_catalog
/bin/sh: line 1: 10257 Segmentation fault: 11  LLVM_PROFILE_FILE=itodbc_catalog.profraw /Users/jordanw/Development/OSSRepos/ODFESQL/sql-odbc/build/odbc/bin/itodbc_catalog
make[3]: *** [IntegrationTests/ITODBCCatalog/CMakeFiles/ccov-run-itodbc_catalog] Error 139
make[2]: *** [IntegrationTests/ITODBCCatalog/CMakeFiles/ccov-run-itodbc_catalog.dir/all] Error 2
make[1]: *** [CMakeFiles/ccov-all.dir/rule] Error 2
make: *** [ccov-all] Error 2

The following line causes the segfault, when bind_tbl[COLUMNS_TYPE_NAME]->AsString() is "geo-point", and is trying to be mapped to an ODBC data type.

auto data_type = data_name_data_type_map
.find(bind_tbl[COLUMNS_TYPE_NAME]->AsString())->second;

This is probably related to Rupal's notes, where:

...information about all columns is retrieved with the query DESCRIBE TABLES LIKE kibana_sample_data_flights. This return all columns including geo-point type which are OriginLocation & DestLocation.

Note: All tests pass when run without coverage.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request SQL
Projects
None yet
Development

No branches or pull requests

4 participants