Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DataX 执行过程中，出错日志信息写入到了tis.log 中，没有打印到控制台执行日志中 #146

Closed
baisui1981 opened this issue Oct 26, 2022 · 1 comment
Closed

DataX 执行过程中，出错日志信息写入到了tis.log 中，没有打印到控制台执行日志中 #146

baisui1981 opened this issue Oct 26, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@baisui1981
Copy link
Member

由于DataX执行过程中出错，但是执行具体的执行日志却打印到了tis.log 日志中，导致没有办法判断具体执行出错信息。

2022-10-26 13:02:07 INFO  c.a.d.c.t.TaskGroupContainer- PerfTrace not enable!
Exception in thread "taskGroup-0" com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-05], Description:[往您配置的写入表中写入数据时失败.].  - java.sql.BatchUpdateException: Data truncation: Data too long for column 'instance_id' at row 1
	at sun.reflect.GeneratedConstructorAccessor12.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
	at com.mysql.jdbc.Util.getInstance(Util.java:408)
	at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1632)
	at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1298)
	at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doBatchInsert(CommonRdbmsWriter.java:543)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:474)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:502)
	at com.alibaba.datax.plugin.writer.mysqlwriter.MysqlWriter$Task.startWrite(MysqlWriter.java:78)
	at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'instance_id' at row 1
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3976)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
	at com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1288)
	at com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:788)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2058)
	at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5158)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1593)
	... 8 more
 - java.sql.BatchUpdateException: Data truncation: Data too long for column 'instance_id' at row 1
	at sun.reflect.GeneratedConstructorAccessor12.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
	at com.mysql.jdbc.Util.getInstance(Util.java:408)
	at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1632)
	at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1298)
	at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doBatchInsert(CommonRdbmsWriter.java:543)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:474)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:502)
	at com.alibaba.datax.plugin.writer.mysqlwriter.MysqlWriter$Task.startWrite(MysqlWriter.java:78)
	at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'instance_id' at row 1
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3976)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
	at com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1288)
	at com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:788)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2058)
	at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5158)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1593)
	... 8 more

	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:40)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:485)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:502)
	at com.alibaba.datax.plugin.writer.mysqlwriter.MysqlWriter$Task.startWrite(MysqlWriter.java:78)
	at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.BatchUpdateException: Data truncation: Data too long for column 'instance_id' at row 1
	at sun.reflect.GeneratedConstructorAccessor12.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
	at com.mysql.jdbc.Util.getInstance(Util.java:408)
	at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1632)
	at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1298)
	at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doBatchInsert(CommonRdbmsWriter.java:543)
	at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:474)
	... 4 more
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'instance_id' at row 1
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3976)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
	at com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1288)
	at com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:788)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2058)
	at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5158)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1593)
	... 8 more
@baisui1981 baisui1981 added bug Something isn't working 3.6.0-alpha labels Oct 26, 2022
@baisui1981
Copy link
Member Author

baisui1981 commented Nov 27, 2022 •
edited
Loading

由于dataX Reader 和Writer运行过程中都是在独立线程中运行的，执行链路是 :

DataXExecutor -> TaskGroupContainer -> WriterRunner/ReaderRunner

需要在 TaskGroupContainer类中添加slfj MDC 上下文设置https://github.com/qlangtech/DataX/blob/master/core/src/main/java/com/alibaba/datax/core/taskgroup/TaskGroupContainer.java#L454-L457

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@baisui1981