New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-21855][YARN] Should print error when upload same file multiple tim… #19073
Conversation
Can one of the admins verify this patch? |
@caneGuy why do you think it is misleading? |
@jerryshao There is a user of our cluster who first saw |
But as I remembered the same name file will be ignored when met again. This should not be a fatal issue, right? |
Job submitting will fail since exception threw.I think this is fatal issue. |
I tried locally with same name jar uploaded twice, the spark application can be started. Can you please paste your exception here? |
|
This is not an error condition. 'someone ignored it' is not sufficient reason. |
@caneGuy can you please check this JIRA (https://issues.apache.org/jira/browse/SPARK-14423), I remembered I fixed this issue before. Can you please first try latest version of Spark, if the issue occurs please describe the step to reproduce this issue in JIRA. Thanks! |
The actual reason i want change the log level is that,if you threw an exception ,then this indicate it is a fatal error,we should not just print a warning. @srowen |
Ok i will try later,thanks @jerryshao |
@jerryshao |
I think it is already in master branch @caneGuy . |
Yes from |
Yes, maybe. You can take a try locally. |
…es to yarn
What changes were proposed in this pull request?
Now when submit job with yarn,and upload same file multiple times.We will throw exception but logging level is warn.
This pr change logging level to error,since warning is misleading.
How was this patch tested?
N/A