Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: MongoDB写入starrocks报错 #1029

Closed
beat4ocean opened this issue Apr 24, 2024 · 2 comments
Closed

[Bug]: MongoDB写入starrocks报错 #1029

beat4ocean opened this issue Apr 24, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@beat4ocean
Copy link

beat4ocean commented Apr 24, 2024

What happened?

addax v4.1.4

当读取mongodb 写入 starrocks:

{
  "job": {
    "setting": {
      "speed": {
        "channel": 3,
        "byte": 1048576
      },
      "errorLimit": {
        "record": 0,
        "percentage": 0.02
      }
    },
    "content": [
      {
        "reader": {
          "name": "mongodbreader",
          "parameter": {
            "connection": {
              "address": [
                "10.81.14.17:20001"
              ],
              "database": "database",
              "collection": "collection",
              "authDb": "admin"
            },
            "username": "username",
            "password": "password",
            "column": [
              {
                "name": "_id",
                "type": "ObjectId"
              },
              {
                "name": "APP_ID",
                "type": "Integer"
              }
            ]
          }
        },
        "writer": {
          "name": "starrockswriter",
          "parameter": {
            "username": "root",
            "password": "password",
            "database": "database",
            "table": "table",
            "column": [
              "`_id`",
              "`app_id`"
            ],
            "jdbcUrl": "jdbc:mysql://172.31.10.40:6033/",
            "loadUrl": [
              "172.31.10.40:8030"
            ],
            "loadProps": {
              "column_separator": "\\x01",
              "row_delimiter": "\\x02"
            }
          }
        }
      }
    ]
  }
}

执行报错

Version

4.1.3 (Default)

OS Type

Linux (Default)

Java JDK Version

Oracle JDK 1.8.0

Relevant log output

Caused by: java.lang.ClassCastException: com.alibaba.fastjson2.JSONObject cannot be cast to java.util.List
	at com.wgzhao.addax.common.util.Configuration.getList(Configuration.java:515)
	at com.wgzhao.addax.plugin.reader.mongodbreader.MongoDBReader$Job.init(MongoDBReader.java:91)
	at com.wgzhao.addax.core.job.JobContainer.initJobReader(JobContainer.java:588)
	at com.wgzhao.addax.core.job.JobContainer.init(JobContainer.java:265)
	at com.wgzhao.addax.core.job.JobContainer.start(JobContainer.java:122)
	... 3 more
@beat4ocean beat4ocean added the bug Something isn't working label Apr 24, 2024
@beat4ocean beat4ocean changed the title [Bug]: 写入starrocks报错 [Bug]: MongoDB写入starrocks报错 Apr 24, 2024
@beat4ocean
Copy link
Author

mongodbreader 这样写一样报错:
"column": [
"_id",
"APP_ID"
]

读取mysql 到 starrocks是正常的

@wgzhao
Copy link
Owner

wgzhao commented Apr 25, 2024

json 文件不正确, connection 对应的值应该是列表,而不是字典,应该写成这样:

"connection": [{
  "address": [ "10.81.14.17:20001"],
  "database": "database",
   "collection": "collection",
  "authDb": "admin"
}],

具体可以参考 https://wgzhao.github.io/Addax/develop/reader/mongodbreader/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants