[Bug] [Worker] Tasks with S3 resource files failed incorrectly

[sketchmind]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

The following error is thrown while executing a task, Failed to download S3(MINIO) resource causing task failed

What you expected to happen

1.The error is because the folder in the resource path is not created
2.The ambiguity of the error log makes it possible to interpret the file as a file on S3, but it is actually the destination file

How to reproduce

Set the resource type to S3 and select a file on S3 in the shell script task

Anything else

No response

Version

dev

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

[SbloodyS]

I don't quite understand your problems. Could you describe it more clearly? @sketchmind

[sketchmind]

I don't quite understand your problems. Could you describe it more clearly? @sketchmind

Just like the shell node in the screenshot above, when the task is executed, the task will first download the dependent file getActiveNN.sh from the S3 file system, but for the resource file path with folders like '/common' above, since the directory '/common' is not created locally in advance, will cause the fail to download.
The temp dir /tmp/dolphinscheduler/exec/process/4262137237344/5999535520736_6/758/4133 created success, but 'common' not in it.

[SbloodyS]

I don't quite understand your problems. Could you describe it more clearly? @sketchmind

Just like the shell node in the screenshot above, when the task is executed, the task will first download the dependent file getActiveNN.sh from the S3 file system, but for the resource file path with folders like '/common' above, since the directory '/common' is not created locally in advance, will cause the download to fail. The temp dir /tmp/dolphinscheduler/exec/process/4262137237344/5999535520736_6/758/4133 created success,but 'common' not in it.

I got it. Do you have time to take a look at this? @guodongym

[guodongym]

Ok, I will take a look