Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mongdbreader 中某些字段可能 不存在的情况时,读数据时,会提示源字段与目标字段个数不一致的问题 #40

Closed
peterchen859 opened this issue Sep 17, 2020 · 2 comments
Labels
enhancement New feature or request

Comments

@peterchen859
Copy link

您好,一直在用您改造的datax,最近有个mongdbreader的问题困绕我,就是mongdb中文档有些字段可能不存在时,配置这个字段后进行读取数据,当读取到没有的字段行时,就提示源数据字段与目标字段数据不一致,请问可否在后续将这个问题解决!

@wgzhao
Copy link
Owner

wgzhao commented Sep 17, 2020

感谢反馈,可以考虑,针对没有字段的行,设置该字段的默认值为null

@wgzhao wgzhao added the enhancement New feature or request label Sep 17, 2020
@wgzhao
Copy link
Owner

wgzhao commented Sep 17, 2020

刚进过测试,当前master分支上的mongodbreader是指支持缺失字段读取为null的功能的。
以下我是的测试结果

2020-09-17 20:27:47.171 [0-0-0-reader] INFO  connection - Opened connection [connectionId{localValue:5, serverValue:30}] to 127.0.0.1:32768
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
add_id	auction_id	new column	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type
unique_id	auction_id	null	content_type

注意到值为new column的这列,实际上我是分两次写入到 mongodb,第一次是有这个字段,值为new column,第二次写入则没有这个值。
我们可以看到最后十条该字段的输出为 null

如果你当前的版本不支持,可以从这里下载我最新版本(提取码 jxxa),替换你当前jar包,然后测试再看看。
如果还有问题,可以贴出你的整个测试过程

@wgzhao wgzhao closed this as completed Sep 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants