Skip to content

Ingest pipelines: JSON processor not honoring field_access_pattern with add_to_root #135170

@flash1293

Description

@flash1293

When using the JSON processor the add_to_root option set, subobject fields are not resolved properly anymore:

POST _ingest/pipeline/_simulate
{
    "docs": [
        {
            "_source": {
                "a": { "b": """{"abchild":"abc"}""" },
                "d.e": """{"dechild":"abc"}"""
            }
        }
    ],
    "pipeline": {
        "processors": [
            {
                "json": {
                    "field": "a.b",
                    "target_field": "abtarget",
                    "ignore_failure": true
                }
            },
            {
                "json": {
                    "field": "d.e",
                    "target_field": "detarget",
                    "ignore_failure": true
                }
            }
        ],
        "field_access_pattern": "flexible"
    }
}

works fine - in case of "field_access_pattern": "classic", detarget is not written successfully, which is expected.

However

POST _ingest/pipeline/_simulate
{
    "docs": [
        {
            "_source": {
                "a": { "b": """{"abchild":"abc"}""" },
                "d.e": """{"dechild":"abc"}"""
            }
        }
    ],
    "pipeline": {
        "processors": [
            {
                "json": {
                    "field": "a.b",
                    "add_to_root": true,
                    "ignore_failure": true
                }
            },
            {
                "json": {
                    "field": "d.e",
                    "add_to_root": true,
                    "ignore_failure": true
                }
            }
        ],
        "field_access_pattern": "flexible"
    }
}

only works for d.e, not for a.b - this is even the case for "field_access_pattern": "classic" or omitting it altogether

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions