Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

添加MongoDB数据源报错,MongoDB版本号为5.01.2 #254

Closed
ZYPCSJ opened this issue Aug 14, 2023 · 5 comments
Closed

添加MongoDB数据源报错,MongoDB版本号为5.01.2 #254

ZYPCSJ opened this issue Aug 14, 2023 · 5 comments
Milestone

Comments

@ZYPCSJ
Copy link

ZYPCSJ commented Aug 14, 2023

java.lang.IllegalStateException
	at com.qlangtech.tis.plugin.ds.mangodb.MangoDBDataSourceFactory.getDbConfig(MangoDBDataSourceFactory.java:142)
	at com.qlangtech.tis.plugin.ds.DataSourceFactory$BaseDataSourceFactoryDescriptor.validateDSFactory(DataSourceFactory.java:432)
	at com.qlangtech.tis.plugin.ds.DataSourceFactory$BaseDataSourceFactoryDescriptor.verify(DataSourceFactory.java:423)
	at com.qlangtech.tis.extension.Descriptor$3.visit(Descriptor.java:621)
	at com.qlangtech.tis.extension.Descriptor$3.visit(Descriptor.java:609)
	at com.qlangtech.tis.extension.impl.RootFormProperties.accept(RootFormProperties.java:75)
	at com.qlangtech.tis.extension.Descriptor.verify(Descriptor.java:609)
	at com.qlangtech.tis.util.AttrValMap.validate(AttrValMap.java:100)
	at com.qlangtech.tis.coredefine.module.action.PluginAction.parsePluginItems(PluginAction.java:869)
	at com.qlangtech.tis.coredefine.module.action.PluginAction.doSavePluginConfig(PluginAction.java:785)
	at sun.reflect.GeneratedMethodAccessor224.invoke(Unknown Source)
@baisui1981
Copy link
Member

收到

@baisui1981 baisui1981 added this to the v3.8.0 milestone Aug 14, 2023
@baisui1981
Copy link
Member

baisui1981 commented Aug 31, 2023

做了如下改造:

  1. 支持 reader端(例如:MySQL库)多表导入到 mongo的多个collection中。
  2. 支持reader端 ,在一个TIS 数据通道中读取多张表。
  3. 支持 Mongo reader中 将某个Document类型的field (使用 jsonpath )将内部的key->val 拆解,flatmap到目标端的多个字段中
    db.user.find()
    [
      {
        _id: ObjectId("64eef6e2f6781f662de4c498"),
        name: 'baisui',
        age: 205,
        profile: { attr1: 1, attr2: 'ddd2' }
      }
    ]
    

例如,以上文档结构,可以导入到mysql中将profile字段内容拆解,最终打平成

  1. name
  2. age
  3. attr1
  4. attr2

@baisui1981
Copy link
Member

baisui1981 commented Sep 1, 2023

@PittYao
Copy link

PittYao commented Sep 1, 2023

mongo cdc 技术文章: https://mp.weixin.qq.com/s/Ls0kv6hPuiruU18W8y9o_g mongo reader datax: https://github.com/alibaba/DataX/blob/master/mongodbreader/doc/mongodbreader.md

添加mongo有同样的问题。3.8.0什么时候发布,期待

@baisui1981
Copy link
Member

mongo cdc 技术文章: https://mp.weixin.qq.com/s/Ls0kv6hPuiruU18W8y9o_g mongo reader datax: https://github.com/alibaba/DataX/blob/master/mongodbreader/doc/mongodbreader.md

添加mongo有同样的问题。3.8.0什么时候发布,期待

预发布的时候 通知你测试,可以留个微信不?@PittYao

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants