datax导入失败

[stgztsw]

datax同步hive的数据到Doris，此时不指定分区，而是指定表路径，datax默认会逐个分区同步，但是会在同步过程中在某个分区报错，也就是部分同步成功。查看报错日志没有errorlog的url，根据日志找到报错的分区，再指定该分区同步，显示同步成功，所以感觉是是同步整个分区表的时候会有问题

com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-05], Description:[往您配置的写入表中写入数据时失败.]. - java.io.IOException: Failed to flush data to doris.
{"BeginTxnTimeMs":0,"CommitAndPublishTimeMs":0,"Label":"my_prefixcc4bbdf4-19e6-424d-ae92-ab2a9effe330_0","LoadBytes":10193,"LoadTimeMs":5,"Message":"wait close failed. tablet writer write failed, tablet_id=613784, txn_id=256869, err=-235, host: 172.19.24.155, host: 172.19.24.156","NumberFilteredRows":0,"NumberLoadedRows":0,"NumberTotalRows":0,"NumberUnselectedRows":0,"ReadDataTimeMs":0,"Status":"Fail","StreamLoadPutTimeMs":1,"TwoPhaseCommit":"false","TxnId":256869,"WriteDataTimeMs":3}
at com.alibaba.datax.plugin.writer.doriswriter.DorisWriterEmitter.doStreamLoad(DorisWriterEmitter.java:114)
at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.flush(DorisWriter.java:123)
at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.startWrite(DorisWriter.java:113)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
at java.lang.Thread.run(Thread.java:748)
- java.io.IOException: Failed to flush data to doris.
{"BeginTxnTimeMs":0,"CommitAndPublishTimeMs":0,"Label":"my_prefixcc4bbdf4-19e6-424d-ae92-ab2a9effe330_0","LoadBytes":10193,"LoadTimeMs":5,"Message":"wait close failed. tablet writer write failed, tablet_id=613784, txn_id=256869, err=-235, host: 172.19.24.155, host: 172.19.24.156","NumberFilteredRows":0,"NumberLoadedRows":0,"NumberTotalRows":0,"NumberUnselectedRows":0,"ReadDataTimeMs":0,"Status":"Fail","StreamLoadPutTimeMs":1,"TwoPhaseCommit":"false","TxnId":256869,"WriteDataTimeMs":3}
at com.alibaba.datax.plugin.writer.doriswriter.DorisWriterEmitter.doStreamLoad(DorisWriterEmitter.java:114)
at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.flush(DorisWriter.java:123)
at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.startWrite(DorisWriter.java:113)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
at java.lang.Thread.run(Thread.java:748)

	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:40) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.startWrite(DorisWriter.java:116) ~[na:na]
	at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
Caused by: java.io.IOException: Failed to flush data to doris.
{"BeginTxnTimeMs":0,"CommitAndPublishTimeMs":0,"Label":"my_prefixcc4bbdf4-19e6-424d-ae92-ab2a9effe330_0","LoadBytes":10193,"LoadTimeMs":5,"Message":"wait close failed. tablet writer write failed, tablet_id=613784, txn_id=256869, err=-235, host: 172.19.24.155, host: 172.19.24.156","NumberFilteredRows":0,"NumberLoadedRows":0,"NumberTotalRows":0,"NumberUnselectedRows":0,"ReadDataTimeMs":0,"Status":"Fail","StreamLoadPutTimeMs":1,"TwoPhaseCommit":"false","TxnId":256869,"WriteDataTimeMs":3}
	at com.alibaba.datax.plugin.writer.doriswriter.DorisWriterEmitter.doStreamLoad(DorisWriterEmitter.java:114) ~[na:na]
	at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.flush(DorisWriter.java:123) ~[na:na]
	at com.alibaba.datax.plugin.writer.doriswriter.DorisWriter$Task.startWrite(DorisWriter.java:113) ~[na:na]
	... 2 common frames omitted

[ChPi]

1.1版本有bug，升级到1.1.1
和参考https://doris.apache.org/zh-CN/docs/faq/data-faq?_highlight=235#q4-tablet-writer-write-failed-tablet_id27306172-txn_id28573520-err-235-or--215-or--238

[JNSimba]

-235 is the import frequency is too fast, you can increase the maxBatchRows/maxBatchByteSize parameter for tuning

[stgztsw]

got it