[SPARK-22467]Added a config to support whether stdout_stream and stderr_stream output to disk#19690
[SPARK-22467]Added a config to support whether stdout_stream and stderr_stream output to disk#1969010110346 wants to merge 1 commit intoapache:masterfrom
stdout_stream and stderr_stream output to disk#19690Conversation
|
Test build #83588 has finished for PR 19690 at commit
|
|
Test build #83590 has finished for PR 19690 at commit
|
|
retest this please |
|
Test build #83621 has finished for PR 19690 at commit
|
|
Test build #83716 has finished for PR 19690 at commit
|
|
retest this please |
|
Test build #83720 has finished for PR 19690 at commit
|
|
Under most conditions the users shouldn't ignore the printed error/warning messages, have you observed much redundant buzz texts? |
|
yea, so we make the default value for this configuration is |
stdout_stream and stderr_stream output to diskstdout_stream and stderr_stream output to disk
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
What changes were proposed in this pull request?
We should add a switch to control the
stdout_streamandstdout_streamoutput to disk.In my environment,due to disk I/O blocking, the
stdout_streamoutput is very slow, so it can not be timely cleaning,and this leads the executor process to be blocked.How was this patch tested?
Added a unit test