running gh-ost on table with json datatype

[lioraltarescu]

Hello , I'm trying to alter a table with json column and a stored virtual column .

My setup is 4 mysql server 8.0.22 , where two of them are comaster and each comaster has a slave . Only one comaster is open to writes.

this is the table

 CREATE TABLE `a` (
  `id` varbinary(16) NOT NULL,
  `idb` varchar(36) CHARACTER SET latin1 COLLATE latin1_swedish_ci GENERATED ALWAYS AS (json_unquote(json_extract(`jsonobj`,_utf8mb4'$._id'))) STORED NOT NULL,
  `jsonobj` json NOT NULL,
  PRIMARY KEY (`id`,`idb`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin ROW_FORMAT=COMPRESSED

My end goal is to modify the virtual stored column and PK but i get a go driver error when I try to add a simple column
here is my gh-ost command

  /usr/bin/gh-ost    \
 --max-load='Threads_running=500'    \
 --critical-load='Threads_running=1000'    \
 --initially-drop-ghost-table    \
 --initially-drop-old-table    \
 --chunk-size=600    \
 --max-lag-millis=3000    \
 --user='ghostuser'   \
 --password='password'   \
 --assume-master-host='writablecomaster'   \
 --throttle-control-replicas='othercomaster'   \
 --host='comasterslave'  \
 --database=a   \
 --table=b   \
 --verbose    \
 --alter="add yeet varchar(20)"    \
 --assume-rbr    \
 --allow-master-master    \
 --cut-over=default    \
 --concurrent-rowcount    \
 --default-retries=4000    \
 --panic-flag-file=/tmp/ghost.panic.flag    \
 --exact-rowcount    \
 --postpone-cut-over-flag-file=/tmp/ghost.postpone.flag 

I get the error in the in the dry run

2021-01-14 12:30:38 INFO Waiting for ghost table to be migrated. Current lag is 0s
2021-01-14 12:30:38 INFO Intercepted changelog state GhostTableMigrated
2021-01-14 12:30:38 INFO Handled changelog state GhostTableMigrated
2021-01-14 12:30:38 INFO Chosen shared unique key is PRIMARY
2021-01-14 12:30:38 INFO Shared columns are tenant_id,entity
2021-01-14 12:30:38 INFO Tearing down inspector
2021-01-14 12:30:38 INFO Tearing down applier
2021-01-14 12:30:38 INFO Tearing down streamer
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x723e6d]

goroutine 1 [running]:
github.com/github/gh-ost/go/sql.(*ColumnList).HasTimezoneConversion(...)
   /go/src/github.com/github/gh-ost/go/sql/types.go:179
github.com/github/gh-ost/go/logic.(*Inspector).inspectOriginalAndGhostTables(0xc00000ec00, 0x801668, 0x14)
   /go/src/github.com/github/gh-ost/go/logic/inspect.go:187 +0xb9d
github.com/github/gh-ost/go/logic.(*Migrator).Migrate(0xc0000c21b0, 0x0, 0x0)
   /go/src/github.com/github/gh-ost/go/logic/migrator.go:361 +0x54e
main.main()
   /go/src/github.com/github/gh-ost/go/cmd/gh-ost/main.go:293 +0x2962

[shlomi-noach]

I'm going to look into this out of interest.

[shlomi-noach]

@lioraltarescu can you please provide some simplified table data so I can create a test case?

[shlomi-noach]

@lioraltarescu please test this binary: https://github.com/openark/gh-ost/suites/1856467417/artifacts/36046306, built by https://github.com/openark/gh-ost/actions/runs/495894244 as part of openark#13

Please make sure to test on replica and not on production

While the new test suite passes. please let me know if this fixes your problem and please validate the data is intact.

[lioraltarescu]

hey @shlomi-noach we tested it and it solved the issue. 10x A lot !!

[shlomi-noach]

Cool. Just as long as you double and triple check data integrity.

I will submit this PR upstream.

[lioraltarescu]

ok , got it . We checked our use case that we ran into a couple of time but we will check other use case that we had in the past to see everything is ok and update you . 10x

[shlomi-noach]

Irrespective of some spam comments on this issue, I recommend closing it, given openark#13 has been merged.

[shlomi-noach]

Please close this issue?