Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TiDB dataX 任务执行问题 #75

Closed
baisui1981 opened this issue Dec 28, 2021 · 1 comment
Closed

TiDB dataX 任务执行问题 #75

baisui1981 opened this issue Dec 28, 2021 · 1 comment
Labels
bug Something isn't working

Comments

@baisui1981
Copy link
Member

baisui1981 commented Dec 28, 2021

存在两个问题:

  1. 解析TiDB列的metadata 有问题,例如在执行TiDB->StarRocks流程中
    映射生成的DDL脚本:
  CREATE TABLE table1
(
    `id`     VARCHAR(-1) NOT NULL,
    `id1`    VARCHAR(-1),
    `id2`    VARCHAR(-1),
    `id3`    VARCHAR(-1),
    `id4`    VARCHAR(-1),
    `id5`    VARCHAR(-1),
    `id6`    VARCHAR(-1),
    `id7`    VARCHAR(-1),
    `id8`    DATE,
    `id9`    VARCHAR(-1),
    `id10`   DECIMAL(-1,0),
    `id11`   VARCHAR(-1),
    `id12`   BIGINT,
    `id13`   BIGINT,
    `id14`   BIGINT,

varchar 的scala length 丢失了

  1. 执行过程中,如果TiDB中为null值则会 抛出异常
2021-12-27 23:13:30 INFO member com.pingcap.tikv.PDClient- Switched to new leader: [leaderInfo: 10.5.8.232:2379]
2021-12-27 23:13:30 ERROR member c.a.d.c.s.p.t.StdoutPluginCollector-
java.lang.NumberFormatException: null
        at java.lang.Long.parseLong(Long.java:552) ~[na:1.8.0_311]
        at java.lang.Long.parseLong(Long.java:631) ~[na:1.8.0_311]
        at com.qlangtech.tis.plugin.datax.TisDataXTiDBReader$Task.createCol(TisDataXTiDBReader.java:182) ~[tis-ds-tidb-plugin.jar:3.2.0]
        at com.qlangtech.tis.plugin.datax.TisDataXTiDBReader$Task.startRead(TisDataXTiDBReader.java:152) ~[tis-ds-tidb-plugin.jar:3.2.0]
        at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) [datax-core-3.2.0.jar:na]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_311]
@baisui1981 baisui1981 added the bug Something isn't working label Dec 28, 2021
@baisui1981
Copy link
Member Author

v3.3.0中已经发布

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant