Skip to content

[Bug] routine load could not resolve jsonpaths #35448

@felixwluo

Description

@felixwluo

Search before asking

  • I had searched in the issues and found no similar issues.

Version

master

What's Wrong?

Routine load could not resolve jsonpaths, "jsonpaths" = "["$.tags.\"a.b\"","$.tags.k2"]" , it is parsed to "jsonpaths": "["$.tags.\"a.b\","$.tags.k2"]", a.b is missing double quotes, is this unreasonable?

What You Expected?

It can parse the value corresponding to a key like a.b in JSON

How to Reproduce?

  1. kafka json data
{
        "tags": {
                "a.b": "test",
                "k2": "k2_value"
        }
} 
  1. table
CREATE TABLE `q1` (
  `k1` varchar(1024) NULL,
  `k2` varchar(1024) NULL
) ENGINE=OLAP
DUPLICATE KEY(`k1`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`k1`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false"
);
  1. routine load task
CREATE ROUTINE LOAD test.test_q1_load ON q1 
COLUMNS(k1,k2)
PROPERTIES
(
    "desired_concurrent_number"="3",
    "max_batch_interval" = "5",
    "max_batch_rows" = "300000",
    "max_batch_size" = "209715200",
    "strict_mode" = "false",
    "format" = "json",
    "jsonpaths" = "[\"$.tags.\\\"a.b\\\"\",\"$.tags.k2\"]",
    "strip_outer_array" = "false",
     "max_error_number"= "10000000000"
)
FROM KAFKA
(
    "kafka_broker_list" = "localhost:9092",
    "kafka_topic" = "test_topic",
    "property.group.id" = "test_group_id"
   
);

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions