Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][core] add parameters in json format #1392

Closed
3 tasks done
aubdiy opened this issue Nov 11, 2022 · 3 comments · Fixed by #1393
Closed
3 tasks done

[Feature][core] add parameters in json format #1392

aubdiy opened this issue Nov 11, 2022 · 3 comments · Fixed by #1393
Labels
feature-request this is a feature requests on the product

Comments

@aubdiy
Copy link
Contributor

aubdiy commented Nov 11, 2022

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

新增个 -pj 参数, 用于接收 json 格式的自定义参数

Use case

  • commond
bin/start-chunjun                              \
    -mode local                                \
    -jobType sync                              \
    -chunjunDistDir chunjun-dist               \
    -job stream.json                           \
    -p a=a1,b=b1                               \
    -pj '{"c":123,"d":"ddd"}'
  • stream.json
{
  "job": {
    "content": [
      {
        "reader": {
          "parameter": {
            "column": [
              {
                "name": "id",
                "type": "id"
              },
              {
                "name": "name",
                "type": "string"
              },
              {
                "name": "content",
                "type": "string"
              },
              {
                "name": "${a}_${b}_${c}_${d}",
                "type": "string"
                
              }
            ],
            "sliceRecordCount": [
              "2"
            ],
            "permitsPerSecond": 1
          },
          "table": {
            "tableName": "sourceTable"
          },
          "name": "streamreader"
        },
        "writer": {
          "parameter": {
            "column": [
              {
                "name": "id",
                "type": "id"
              },
              {
                "name": "name",
                "type": "string"
              }
            ],
            "print": true
          },
          "table": {
            "tableName": "sinkTable"
          },
          "name": "streamwriter"
        },
        "transformer": {
          "transformSql": "select id,name from sourceTable where CHAR_LENGTH(name) < 50 and CHAR_LENGTH(content) < 50"
        }
      }
    ],
    "setting": {
      "errorLimit": {
        "record": 100
      },
      "speed": {
        "bytes": 0,
        "channel": 1,
        "readerChannel": 1,
        "writerChannel": 1
      }
    }
  }
}

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@aubdiy aubdiy added the feature-request this is a feature requests on the product label Nov 11, 2022
@FlechazoW
Copy link
Member

-p 和 -pj 是做同一个事情的,用一个就好了。-p '{"c":123,"d":"ddd"}' 这样会不会更好一些呢?

@aubdiy
Copy link
Contributor Author

aubdiy commented Nov 11, 2022

'{"c":123,"d":"ddd"}' 这样会不会更好一些呢?

为了兼容历史任务, 不然升级改动太大了.
也可以在处理 '-p' 参数内容时, 根据解析 json 异常与否, 来兼容之前的写法,
但是如果这样的话, 这个兼容就永远无法下线了,
最终,
还是觉得新增一个 -pj 参数接收 json 格式数据,
-p 标记为即将失效,
然后再未来某个合适的时机, 去掉原来的 -p 功能

@FlechazoW
Copy link
Member

了解,那先做成-pj 的方式来表示是否是json内容,在下个大版本中对这部分做调整优化。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request this is a feature requests on the product
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants