Skip to content

[BUG] Nested fields of generated structs not accessible #4682

@yuancu

Description

@yuancu

Query Information

PPL Command/Query:

source=any | head 1 | eval ip='1.2.3.4' | eval info = geoip('my-datasource', ip)

Expected Result:

{
  "schema": [
    {
      "name": "info.continent_name",
      "type": "string"
    }
  ],
  "datarows": [
    [
      "Oceania"
    ]
  ],
  "total": 1,
  "size": 1
}

Actual Result:

{
  "schema": [
    {
      "name": "$f0",
      "type": "undefined"
    }
  ],
  "datarows": [
    [
      null
    ]
  ],
  "total": 1,
  "size": 1
}
  1. Value is not parsed
  2. Field name is incorrect
  3. Type is incorrect

Dataset Information

Dataset/Schema Type

  • Custom (details below)

Index Mapping

{
  "mappings": {
    "properties": {
      "host": {
        "type": "ip"
      }
    }
  }
}

Sample Data

{"host":"1.2.3.4"}

Bug Description

Issue Summary:
Nested fields of generated structs are not accessible.

In comparison, those of inbuilt ones (instead of generated, the nested field comes with a mapping) are accessible. E.g. for otellogs, the following PPL runs without a problem

source=opensearch-sql_test_index_otel_logs | fields resource.attributes.service.name | head 1
{
  "schema": [
    {
      "name": "resource.attributes.service.name",
      "type": "string"
    }
  ],
  "datarows": [
    [
      "security-scanner"
    ]
  ],
  "total": 1,
  "size": 1
}

Steps to Reproduce:

  1. Configure IP datasource according to this doc: https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/functions/ip.rst#geoip
  2. run the above PPL

Impact:

  • Cannot access nested fields of generated objects

Environment Information

OpenSearch Version:
3.4.0

Metadata

Metadata

Assignees

Labels

PPLPiped processing languagebugSomething isn't working

Type

No type

Projects

Status

Not Started

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions