VReplication: ignore generated columns in workflows

[rohit-nayak-ps]

Description

Currently generated columns are not supported during a resharding workflow. MySQL does not allow values to be provided for generated columns either for insert or update. Today, generated columns are treated as regular columns and hence they attempt to populate such columns.

This PR identifies these columns by looking at the Extra column in information_schema.columns and drops them from insert/update statements on the target.

Note: We continue to send these columns as part of the vstreamer events so the downstream consumers will see the actual values. We need to do this since vstreams might be consumed by CDC pipelines which should be expected to recompute the generated column data.

Checklist

• Tests were added or are not required
• Documentation was added or is not required

[shlomi-noach]

initial comments, not thorough review.

[shlomi-noach]

this becomes very parallel to

vitess/go/vt/vttablet/onlineddl/vrepl/types.go

Lines 35 to 58 in 50365bb

	const (
	UnknownColumnType ColumnType = iota
	TimestampColumnType
	DateTimeColumnType
	EnumColumnType
	MediumIntColumnType
	JSONColumnType
	FloatColumnType
	BinaryColumnType
	)
	// Column represents a table column
	type Column struct {
	Name string
	IsUnsigned bool
	Charset string
	Type ColumnType
	EnumValues string
	EnumToTextConversion bool
	// add Octet length for binary type, fix bytes with suffix "00" get clipped in mysql binlog.
	// https://github.com/github/gh-ost/issues/909
	BinaryOctetLength uint64
	}

. I see we are solving the same kind of problems in different places. I'll defer to yours anyway.

[shlomi-noach]

map[string][]*ColumnInfo confuses me. Is the key a table-name? Let's consider making a new type to simplify the syntax.

[systay]

I sometimes use type aliases for this.

type tableNameStr = string
colInfoMap := map[tableNameStr][]*ColumnInfo{}

[rohit-nayak-ps]

This is a great pattern. There are quite a few places where such maps are used it this will increase code clarity. I will fix it up when I touch this module next since likely it will affect quite a few lines of code and also tests might need to be updated.