Data type macros

[jtcohen6]

See dbt-labs/dbt-core#5428

[jtcohen6]

TestTypeInt and TestTypeFloat were failing because we infer/load numeric columns with decimals as double and without decimals as bigint.

dbt-spark/dbt/adapters/spark/impl.py

Lines 92 to 95 in 48e1989

	@classmethod
	def convert_number_type(cls, agate_table, col_idx):
	decimals = agate_table.aggregate(agate.MaxPrecision(col_idx))
	return "double" if decimals else "bigint"

Whereas the type macros are returning float and int, respectively. All the more reason to reconcile between these convert_{X}_type methods (for agate use), and the values returned by Column.translate_type (dbt-labs/dbt-core#5317). We could accomplish that within this adapter, by setting up type translation for SparkColumn from int → bigint and float → double.

For now, skip inference and just set the types we want in each seed.

[dbeatty10]

👍

[dbeatty10]

That is unfortunate behavior.

[jtcohen6]

Agreed — the distinction between = and EXCEPT doesn't make much sense to me, but I'm sure there is a reason (or was, at some point)