Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [seatunnel job ] When executing the local seatunneljob, an error message is displayed indicating that the file cannot be found #15156

Closed
3 tasks done
Winn-bmt opened this issue Nov 13, 2023 · 1 comment · Fixed by #15621
Labels
bug Something isn't working priority:high

Comments

@Winn-bmt
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

将资源中心配置在本地时,对于上传到资源中的脚本文件,在执行seatunnel任务脚本,显示找不到脚本文件。因为在dolphinscheduler-task-plugin/dolphinscheduler-task-seatunnel/src/main/java/org/apache/dolphinscheduler/plugin/task/seatunnel/SeatunnelTask.java`文件的代码中,对ResourceName进行切分时,只切割了1个字符,代码如下

    protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                args.add(resourceInfo.getResourceName().substring(1));
            });
        }
        return args;
    }

所以在初始化时,生成的脚本文件路径字符串前缀是file:,只切割掉1个字符,会导致文件路径包含ile: ,进而生成错误的路径字符串ile:/tmp/dolphinscheduler

所以实际应该切割掉5个字符。只切割掉一个字符会报如下的错误:
Snipaste_2023-11-13_15-54-11
Snipaste_2023-11-13_15-55-40

希望可以尽快处理一下,只需要在 args.add(resourceInfo.getResourceName().substring(1));改为args.add(resourceInfo.getResourceName().substring(5));
希望在下个版本中可以进行修复,谢谢!

What you expected to happen

Snipaste_2023-11-13_15-54-11 Snipaste_2023-11-13_15-55-40

How to reproduce

向资源中心上传seatunnel的作业文件,进行调度执行,会报错 找不到作业文件

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@Winn-bmt Winn-bmt added bug Something isn't working Waiting for reply Waiting for reply labels Nov 13, 2023
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

When the resource center is configured locally, for the script file uploaded to the resource, when executing the seatunnel task script, it shows that the script file cannot be found. Because in the code of the dolphinscheduler-task-plugin/dolphinscheduler-task-seatunnel/src/main/java/org/apache/dolphinscheduler/plugin/task/seatunnel/SeatunnelTask.java` file, when splitting the ResourceName, only One character is cut, the code is as follows

    protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                args.add(resourceInfo.getResourceName().substring(1));
            });
        }
        return args;
    }

Therefore, during initialization, the prefix of the generated script file path string is file:, and only one character is cut off, which will cause the file path to contain ile:, thus generating the wrong path string ile:/tmp/dolphinscheduler

So actually 5 characters should be cut off. Cutting off only one character will result in the following error:
Snipaste_2023-11-13_15-54-11
Snipaste_2023-11-13_15-55-40

I hope it can be dealt with as soon as possible, just change args.add(resourceInfo.getResourceName().substring(1)); to args.add(resourceInfo.getResourceName().substring(5));
Hope it can be fixed in the next version, thank you!

What you expected to happen

Snipaste_2023-11-13_15-54-11 Snipaste_2023-11-13_15-55-40

How to reproduce

Upload the seatunnel job file to the resource center for scheduling and execution. An error will be reported. The job file cannot be found.

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@Winn-bmt Winn-bmt changed the title [Bug] [Module Name] Bug title [Bug] [seatunnel job ] When executing the local seatunneljob, an error message is displayed indicating that the file cannot be found Nov 13, 2023
@zhongjiajie zhongjiajie added this to the 3.2.1 milestone Jan 30, 2024
@SbloodyS SbloodyS removed the Waiting for reply Waiting for reply label Feb 23, 2024
@SbloodyS SbloodyS removed this from the 3.2.1 milestone Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working priority:high
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants