clickhouse_loader.py : fixing temporal and binary data types

[aadant]

This fixes the checksums for temporal types for timezone != UTC

MySQL timestamps are converted to DateTime64 in the client timezone.
MySQL DateTime are converted as String, so that they are converted in the server timezone.

For the migration to be consistent, the source and target timezones should be identical. It should not matter for timestamps but it could cause conversion issues for DateTime.

time python db_load/clickhouse_loader.py --clickhouse_host $CH_HOST--clickhouse_database data_types --dump_dir /home/aadant/dbdumps/data_types --clickhouse_user $CH_USER --clickhouse_password $CH_PASS --clickhouse_port 9000 --threads 16  --mysql_source_database data_types --mysqlshell  --rmt_delete_support

test : 

python db_compare/mysql_table_checksum.py --mysql_host $MYSQL_HOST --mysql_user $MYSQL_USER --mysql_password $MYSQL_PASS --mysql_database $DATABASE --tables_regex . --threads 4 | grep "Checksum for table" | awk '{print $11" "$13" "$15}' | sort > mysql

python db_compare/clickhouse_table_checksum.py --clickhouse_host $CH_HOST --clickhouse_user $CH_USER --clickhouse_password $PASS  --clickhouse_database data_types --tables_regex . --threads 4  --sign_column "" | grep "Checksum for table" | awk '{print $11" "$13" "$15}' | sort >ch

diff ch mysql

The binary types have been addressed. Binary types are encoded as base64 by clickhouse_loader.py, while the checksum encoding is currently hex only (see for the sink-connector FR #340)

[subkanthi]

it will be nice if these values 2299-12-31 are in a separate file or a separate variable , so that it can be updated easily.

[aadant]

good point, this was tested with 23.3. @subkanthi do you know if there is a JDBC driver that supports those maximum values ?
will add it as a parameter as it may change again

[aadant]

@subkanthi Added --min_date_value and --max_date_value, please review !