Table Definitions

Kinesis streams stores data as stream of bytes and leaves it to produceres and consumers to define how a message should be interpreted. For Presto, this data must be mapped into columns to allow queries against the data.

A table definition file consists of a JSON definition for a table. The name of the file can be arbitrary but must end in .json.

{
    "tableName": ...,
    "schemaName": ...,
    "streamName": ...,
    "message": {
        "dataFormat": ...,
        "field": [
            ...
        ]
    }
}

Field	Required	Type	Description
tableName	required	string	Presto table name defined by this file
schemaName	optional	string	Schema which will contain the table. If omitted, the default schema name is used
streamName	required	string	Kinesis Stream that is mapped
message	optional	JSON object	Field definitions for data columns mapped to the message itself

Every message in Kinesis stream can be decoded using the table definition file. The json object message in table definition file contains two fields:

Field	Required	Type	Description
dataFormat	required	string	Selects the decoder for this group of fields
fields	required	JSON array	A list of field definitions. Each field definition creates a new column in the Presto table

Each field definition is a JSON object:

{
    "name": ...,
    "type": ...,
    "dataFormat": ...,
    "mapping": ...,
    "formatHint": ...,
    "hidden": ...,
    "comment": ...
}

Field	Required	Type	Description
name	required	string	Name of the column in the Presto table
type	required	string	Presto type of the column
dataFormat	optional	string	Selects the column decoder for this field. Default to the default decoder for this row data format and
column type
mapping	optional	string	Mapping information for the column. This is decoder specific, see below
formatHint	optional	string	Sets a column specific format hint to the column decoder
hidden	optional	boolean	Hides the column from DESCRIBE <table name> and SELECT *. Defaults to false
comment	optional	string	Add a column comment which is shown with DESCRIBE <table name>

There is no limit on field descriptions for message.