[Bug] [Doris 2 Doris] Unsupported doris type: JSON

[oldwolf1984]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

编写脚本，读取Doris源库中一个数据表，数据表中一个名叫data的字段，类型是json类型，期望读取data.属性1，写入Doris另一个目标表中，目标表是一个普通字符串类型字段

SeaTunnel Version

2.3.5

SeaTunnel Config

无

Running Command

执行脚本

Error Exception

2024-06-10 14:19:37,125 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Doris'.
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
	... 2 more
Caused by: org.apache.seatunnel.api.table.catalog.exception.CatalogException: ErrorCode:[API-03], ErrorDescription:[Catalog initialize failed] - Unsupported doris type: JSON
	at org.apache.seatunnel.connectors.doris.util.DorisCatalogUtil.fromDorisType(DorisCatalogUtil.java:260)
	at org.apache.seatunnel.connectors.doris.catalog.DorisCatalog.getTable(DorisCatalog.java:233)
	at org.apache.seatunnel.connectors.doris.source.DorisSourceFactory.createSource(DorisSourceFactory.java:86)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:111)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)
	... 7 more
 
2024-06-10 14:19:37,125 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 
===============================================================================



Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Doris'.
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
	at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
	at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
	... 2 more
Caused by: org.apache.seatunnel.api.table.catalog.exception.CatalogException: ErrorCode:[API-03], ErrorDescription:[Catalog initialize failed] - Unsupported doris type: JSON
	at org.apache.seatunnel.connectors.doris.util.DorisCatalogUtil.fromDorisType(DorisCatalogUtil.java:260)
	at org.apache.seatunnel.connectors.doris.catalog.DorisCatalog.getTable(DorisCatalog.java:233)
	at org.apache.seatunnel.connectors.doris.source.DorisSourceFactory.createSource(DorisSourceFactory.java:86)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:111)
	at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Carl-Zhou-CN]

@oldwolf1984 dev has enhancements to doris's type that you can try

[EricJoy2048]

Hi, In seatunnel 2.3.5, the doris source connector support read json column. And you can get data.XBNO use SQL Transform.

The job config can like this:

source {
  Doris{
    xxx
    doris.read.field = "id, data"

  }
}

transform{
  Sql {
    query = "select id, id as oid, data.XBNO as materialNo from book_cms_dwt"
  }
}

Sink {
  xxx
}