[Bug] [minio] create tenant error

[jealone]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

I want use the s3/minio as the resources storage, but when I deploy the cluster, I have failed to create tenant. And I get the error logs as follow

ERROR] 2021-11-02 14:12:36.262 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[45] - 创建租户错误
org.apache.hadoop.fs.FileAlreadyExistsException: Path is a file: /beeto_bi/resources
	at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:853)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1881)
	at org.apache.dolphinscheduler.common.utils.HadoopUtils.mkdir(HadoopUtils.java:276)
	at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:129)
	at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120)
	at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
	at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$46064c30.createTenant(<generated>)
	at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85)
.....

And then I scan the source code，I find that tenant Controller will mkdir with udf and resources，but S3AFilesystem in hadoop-aws-2.7.3 will check if the path is directory. Unfortunately, the check rule is that if the path is end with '/'.

any instructions details to support minio?

What you expected to happen

use minio as the resources storage

How to reproduce

memention above

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

I want use the s3/minio as the resources storage, but when I deploy the cluster, I have failed to create tenant. And I get the error logs as follow

ERROR] 2021-11-02 14:12:36.262 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[45] - 创建租户错误
org.apache.hadoop.fs.FileAlreadyExistsException: Path is a file: /beeto_bi/resources
	at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:853)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1881)
	at org.apache.dolphinscheduler.common.utils.HadoopUtils.mkdir(HadoopUtils.java:276)
	at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:129)
	at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120)
	at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
	at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$46064c30.createTenant(<generated>)
	at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85)
.....

And then I scan the source code，I find that tenant Controller will mkdir with udf and resources，but S3AFilesystem in hadoop-aws-2.7.3 will check if the path is directory. Unfortunately, the check rule is that if the path is end with '/'.

any instructions details to support minio?

What you expected to happen

use minio as the resources storage

How to reproduce

memention above

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Hi:

• Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.