[BEAM-6604] Check for null values when encoding Key columns.

[nielm]

When encoding Key Column values, if the column value is unspecified, assume that the value is null.

This corrects an NPE when encoding keys.
@chamikaramj

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---	---	---	---	---
Java
Python		---			---	---	---

[chamikaramj]

So basically we consider all fields that are omitted as NULL ? Isn't it better to ask users to explicitly set these values to NULL ? I'm worried if this will result in NULL values being inserted due to bugs in user code.

Another option might be to make this more explicit, for example, by adding a transform builder method withNullAsDefault().

Also, looks like Spanner supports NOT NULL columns which can result in a confusing behavior in-combination with a default non-explicit NULL from Beam.

WDYT ?

[nielm]

Consider the behavior of a normal insert statement that omits a column value:

CREATE TABLE test (
    key1 STRING(MAX), 
    key2 STRING(max)) 
    PRIMARY KEY (key1, key2)
INSERT INTO test 
    (key1) 
    VALUES ("a")

This will result in a row with values ("a", <NULL>)
So assuming a missed value == NULL is the same behavior as the database.

Note also that this code is only concerned with Primary Key columns, and the encoded value is only used for sorting mutations in a bundle by table and primary key -- the value encoded here is never written to the database.

NOT NULL constraints can be set on any column. If the Mutation is an insert and does not specify a column which is NOT NULL (or sets it to NULL), then the insert will fail, and the Mutation appended to a PCollection of failed MutationGroups which is output by the SpannerIO.Write transform.

[chamikaramj]

Ok. Thanks for clarifying.

[chamikaramj]

Run Beam PostCommit

[chamikaramj]

Run Java PostCommit

[chamikaramj]

LGTM. Will merge after post-commit tests pass.