Skip to content

[External Stream] Format settings declared on creation time should be taken into account #533

@zliang-min

Description

@zliang-min

Use case

Currently, format settings can be declared on creation time when create a external stream, for example:

create or replace external stream users (
           user_id uint32,
           message string,
           timestamp datetime,
           metric float32
           ) settings type='kafka',
           topic='src',
           brokers='localhost:9092',
           data_format='JSONEachRow',
           input_format_skip_unknown_fields=true;

The above code uses input_format_skip_unknown_fields=true and expects SELECT queries will ignore unknow fileds. However this does not work. One has to specify the setting in the SELECT queries like:

SELECT * FROM users SETTINGS  input_format_skip_unknown_fields=true

This is troublesome, because such settings likely should be applied to every SELECT queries against that stream.

Describe the solution you'd like

The format settings declared on creation time should be taken into account.

Describe alternatives you've considered

Additional context

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions