Skip to content

Support multiline directives in connection file parser #10

@gnzjgo

Description

@gnzjgo

Problem

The connection file parser does not support multiline directives using the > block syntax. This is needed for KAFKA_SSL_CA_PEM when providing SSL/TLS certificates in PEM format (e.g. for Aiven Kafka or self-signed certs).

A connection file like this fails to parse:

TYPE kafka
KAFKA_BOOTSTRAP_SERVERS localhost:9092
KAFKA_SSL_CA_PEM >
    -----BEGIN CERTIFICATE-----
    MIIDXTCCAkWgAwIBAgIJAM...
    -----END CERTIFICATE-----

The parser iterates line-by-line and treats the indented certificate lines as unknown directives, raising MigrationParseError.

Solution

  • Update the connection parser to use an index-based loop and read_directive_block() (already used by datasource/pipe parsers) to handle KAFKA_SSL_CA_PEM > multiline blocks.
  • Update the connection generator to emit the > block syntax when ssl_ca_pem contains newlines.
  • Single-line KAFKA_SSL_CA_PEM values (e.g. {{ tb_secret(...) }}) continue to work as before.

Ref: tinybirdco/tinybird-sdk-typescript#129

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions