Skip to content

Commit

Permalink
DBZ-866 Updating docs of 'source' block for MySQL connector
Browse files Browse the repository at this point in the history
  • Loading branch information
@gunnarmorling @jpechane
gunnarmorling authored and jpechane committed Aug 21, 2018
1 parent eebb97f commit e77b8c9
Showing 1 changed file with 58 additions and 8 deletions.
66 changes: 58 additions & 8 deletions docs/connectors/mysql.asciidoc
View file
Expand Up @@ -340,7 +340,7 @@ Meanwhile, the schema change event message's value will contain a structure cont
"fields": [
{
"type": "string",
"optional": false,
"optional": true,
"field": "version"
},
{
Expand Down Expand Up @@ -381,7 +381,28 @@ Meanwhile, the schema change event message's value will contain a structure cont
{
"type": "boolean",
"optional": true,
"default": false,
"field": "snapshot"
},
{
"type": "int64",
"optional": true,
"field": "thread"
},
{
"type": "string",
"optional": true,
"field": "db"
},
{
"type": "string",
"optional": true,
"field": "table"
},
{
"type": "string",
"optional": true,
"field": "query"
}
]
}
Expand All @@ -399,7 +420,11 @@ Meanwhile, the schema change event message's value will contain a structure cont
"file": "mysql-bin.000003",
"pos": 154,
"row": 0,
"snapshot": true
"snapshot": true,
"thread": null,
"db": null,
"table": null,
"query": null
}
}
}
Expand Down Expand Up @@ -493,7 +518,7 @@ The value of the change event message is a bit more complicated. Like the key me
* `op` is a mandatory field that contains a string value describing the type of operation. Values for the MySQL connector are `c` for create (or insert), `u` for update, `d` for delete, and `r` for read (in the case of a non-initial snapshot).
* `before` is an optional field that if present contains the state of the row _before_ the event occurred. The structure will be described by the `mysql-server-1.inventory.customers.Value` Kafka Connect schema, which the `mysql-server-1` connector uses for all rows in the `inventory.customers` table.
* `after` is an optional field that if present contains the state of the row _after_ the event occurred. The structure is described by the same `mysql-server-1.inventory.customers.Value` Kafka Connect schema used in `before`.
* `source` is a mandatory field that contains a structure describing the source metadata for the event, which in the case of MySQL contains several fields: the Debezium version, the connector name, the name of the binlog file where the event was recorded, the position in that binlog file where the event appeared, the row within the event (if there is more than one), whether this event was part of a snapshot, and if available the MySQL server ID, and the timestamp in seconds. For non-snapshot events, if the MySQL server has the link:#enabling-query-log-events-optional[binlog_rows_query_log_events] option enabled, and the connector is configured with the `include.query` option enabled, the query field will contain the original SQL statement that generated the event.
* `source` is a mandatory field that contains a structure describing the source metadata for the event, which in the case of MySQL contains several fields: the Debezium version, the connector name, the name of the binlog file where the event was recorded, the position in that binlog file where the event appeared, the row within the event (if there is more than one), whether this event was part of a snapshot, name of the affected database and table, id of the MySQL thread creating the event (non-snapshot events only), and if available the MySQL server ID, and the timestamp in seconds. For non-snapshot events, if the MySQL server has the link:#enabling-query-log-events-optional[binlog_rows_query_log_events] option enabled, and the connector is configured with the `include.query` option enabled, the query field will contain the original SQL statement that generated the event.
* `ts_ms` is optional and if present contains the time (using the system clock in the JVM running the Kafka Connect task) at which the connector processed the event.

And of course, the _schema_ portion of the event message's value contains a schema that describes this envelope structure and the nested fields within it.
Expand Down Expand Up @@ -579,7 +604,7 @@ Let's look at what a _create_ event value might look like for our `customers` ta
"fields": [
{
"type": "string",
"optional": false,
"optional": true,
"field": "version"
},
{
Expand Down Expand Up @@ -620,13 +645,29 @@ Let's look at what a _create_ event value might look like for our `customers` ta
{
"type": "boolean",
"optional": true,
"default": false,
"field": "snapshot"
},
{
"type": "int64",
"optional": true,
"field": "thread"
},
{
"type": "string",
"optional": true,
"field": "query"
"field": "db"
},
{
"type": "string",
"optional": true,
"field": "table"
},
{
"type": "string",
"optional": true,
"field": "query"
}
]
},
{
Expand Down Expand Up @@ -655,7 +696,10 @@ Let's look at what a _create_ event value might look like for our `customers` ta
"file": "mysql-bin.000003",
"pos": 154,
"row": 0,
"snapshot": null,
"snapshot": false,
"thread": 7,
"db": "inventory",
"table": "customers",
"query": "INSERT INTO customers (first_name, last_name, email) VALUES ('Anne', 'Kretchmar', 'annek@noanswer.org')"
}
}
Expand Down Expand Up @@ -707,7 +751,10 @@ Here's that new event's _value_ formatted to be easier to read:
"file": "mysql-bin.000003",
"pos": 484,
"row": 0,
"snapshot": null,
"snapshot": false,
"thread": 7,
"db": "inventory",
"table": "customers",
"query": "UPDATE customers SET first_name='Anne Marie' WHERE id=1004"
},
"op": "u",
Expand Down Expand Up @@ -755,7 +802,10 @@ So far we've seen samples of _create_ and _update_ events. Now, let's look at th
"file": "mysql-bin.000003",
"pos": 805,
"row": 0,
"snapshot": null,
"snapshot": false,
"thread": 7,
"db": "inventory",
"table": "customers",
"query": "DELETE FROM customers WHERE id=1004"
},
"op": "d",
Expand Down

0 comments on commit e77b8c9

Please sign in to comment.