Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InfluxDBWriter 多channel设置无效。 #129

Closed
covered opened this issue Feb 22, 2021 · 1 comment · Fixed by #130
Closed

InfluxDBWriter 多channel设置无效。 #129

covered opened this issue Feb 22, 2021 · 1 comment · Fixed by #130
Labels
bug Something isn't working

Comments

@covered
Copy link

covered commented Feb 22, 2021

配置内容:

"job": {
  "setting": {
    "speed": {
      "channel": **5,**
      //"record": 200,
      "bytes": -1

错误信息:

2021-02-22 08:21:00.688 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2021-02-22 08:21:00.688 [main] INFO JobContainer - DataX jobContainer starts job.
2021-02-22 08:21:00.691 [main] INFO JobContainer - Set jobId = 0
2021-02-22 08:21:00.711 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] do prepare work .
2021-02-22 08:21:00.711 [job-0] INFO JobContainer - DataX Writer.Job [influxdbwriter] do prepare work .
2021-02-22 08:21:10.613 [job-0] INFO JobContainer - Job set Channel-Number to 3 channels.
2021-02-22 08:21:10.614 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] splits to [3] tasks.
2021-02-22 08:21:10.615 [job-0] WARN InfluxDBWriter$Job - -------------------------split() begin...
2021-02-22 08:21:10.615 [job-0] INFO JobContainer - DataX Writer.Job [influxdbwriter] splits to [1] tasks.
2021-02-22 08:21:10.620 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2021-02-22 08:21:10.621 [job-0] ERROR Engine - Code:[Framework-15], Description:[DataX插件切分出错, 该问题通常是由于DataX各个插件编程错误引起,请联系DataX开发团队解决]. - Code:[Framework-15], Description:[DataX插件切分出错, 该问题通常是由于DataX各个插件编程错误引起,请联系DataX开发团队解决]. - reader切分的task数目[3]不等于writer切分的task数目[1].

代码修改:
// @OverRide
// public List split(int adviceNumber)
// {
// Configuration readerSliceConfig = super.getPluginJobConf();
// List splittedConfigs = new ArrayList<>();
// splittedConfigs.add(readerSliceConfig);
// return splittedConfigs;

    // }

    @Override
    public List<Configuration> split(int mandatoryNumber)
    {
        //修改多channel无效问题
        LOG.warn("-------------------------split [{}] begin...", mandatoryNumber);
        ArrayList<Configuration> configurations = new ArrayList<Configuration>(mandatoryNumber);
        for (int i = 0; i < mandatoryNumber; i++) {
                 configurations.add(this.originalConfig.clone());
             }
        return configurations;

    }

初步测试设置多channel后无报错。

@wgzhao wgzhao added the bug Something isn't working label Feb 22, 2021
wgzhao added a commit that referenced this issue Feb 22, 2021
@wgzhao
Copy link
Owner

wgzhao commented Feb 22, 2021

感谢反馈并提供修复代码,已经修复

wgzhao added a commit that referenced this issue Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants