Suggestion: Increase max_allowed_packet for orca-redis-to-sql migration

[shubhamc183]

While migrating the Orca data from Redis to Orca I faced this issue

Packet for query is too large (309,668,956 > 4,194,304). You can change this value on the server by setting the 'max_allowed_packet' variable.

It was also faced by a couple of other folks too while found in Spinnaker's Slack channel. Does it make sense to add a note of this in the orca-redis-to-sql migration docs asking upfront to increase the max_allowed_packet to a sane value?

2021-05-30 17:18:00.182 ERROR 1 --- [igrationAgent-0] n.s.o.n.AbstractPollingNotificationAgent : [] Error running agent tick
org.springframework.dao.TransientDataAccessResourceException: jOOQ; SQL [insert into pipeline_stages (id, legacy_id, execution_id, status, updated_at, body) values (?, ?, ?, ?, ?, ?) on duplicate key update status = ?, updated_at = ?, body = ? -- agentClass: PipelineMigrationAgent]; Packet for query is too large (309,668,956 > 4,194,304). You can change this value on the server by setting the 'max_allowed_packet' variable.; nested exception is com.mysql.cj.jdbc.exceptions.PacketTooBigException: Packet for query is too large (309,668,956 > 4,194,304). You can change this value on the server by setting the 'max_allowed_packet' variable.
	at org.jooq_3.13.2.MYSQL.debug(Unknown Source)
	at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:110)
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
	at com.netflix.spinnaker.kork.sql.JooqToSpringExceptionTransformer.exception(JooqToSpringExceptionTransformer.kt:36)
	at org.jooq.impl.ExecuteListeners.exception(ExecuteListeners.java:274)
	at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:386)
	at org.jooq.impl.AbstractDelegatingQuery.execute(AbstractDelegatingQuery.java:119)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.upsert(SqlExecutionRepository.kt:931)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.storeStageInternal(SqlExecutionRepository.kt:871)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.storeStageInternal$default(SqlExecutionRepository.kt:846)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.storeExecutionInternal(SqlExecutionRepository.kt:832)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.access$storeExecutionInternal(SqlExecutionRepository.kt:84)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository$store$$inlined$withPool$lambda$1.invoke(SqlExecutionRepository.kt:133)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository$store$$inlined$withPool$lambda$1.invoke(SqlExecutionRepository.kt:84)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository$transactional$1$1.run(SqlExecutionRepository.kt:1018)
	at org.jooq.impl.DefaultDSLContext$3.run(DefaultDSLContext.java:636)
	at org.jooq.impl.DefaultDSLContext$3.run(DefaultDSLContext.java:633)
	at org.jooq.impl.DefaultDSLContext.lambda$transactionResult0$0(DefaultDSLContext.java:564)
	at org.jooq.impl.Tools$12$1.block(Tools.java:4890)
	at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128)
	at org.jooq.impl.Tools$12.get(Tools.java:4887)
	at org.jooq.impl.DefaultDSLContext.transactionResult0(DefaultDSLContext.java:616)
	at org.jooq.impl.DefaultDSLContext.transactionResult(DefaultDSLContext.java:533)
	at org.jooq.impl.DefaultDSLContext.transaction(DefaultDSLContext.java:633)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository$transactional$1.get(SqlExecutionRepository.kt:1017)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository$transactional$1.get(SqlExecutionRepository.kt:84)
	at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:34)
	at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:27)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.transactional(SqlExecutionRepository.kt:1015)
	at com.netflix.spinnaker.orca.sql.pipeline.persistence.SqlExecutionRepository.store(SqlExecutionRepository.kt:133)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.netflix.spinnaker.kork.telemetry.InstrumentedProxy.invoke(InstrumentedProxy.java:103)
	at com.sun.proxy.$Proxy145.store(Unknown Source)
	at com.netflix.spinnaker.orca.pipeline.persistence.migration.PipelineMigrationAgent.tick(PipelineMigrationAgent.kt:64)
	at com.netflix.spinnaker.orca.notifications.AbstractPollingNotificationAgent.lambda$startPolling$0(AbstractPollingNotificationAgent.java:63)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.mysql.cj.jdbc.exceptions.PacketTooBigException: Packet for query is too large (309,668,956 > 4,194,304). You can change this value on the server by setting the 'max_allowed_packet' variable.
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:107)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
	at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:370)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
	at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:209)
	at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:453)
	at org.jooq.impl.AbstractDMLQuery.execute(AbstractDMLQuery.java:904)
	at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:371)