[SPARK-26694][CORE] Progress bar should be enabled by default for spark-shell #23618
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rk-shell SPARK-21568 made a change to ensure that progress bar is enabled for spark-shell by default but not for other apps. Before that change, this was distinguished using log-level which is not a good way to determine the same as users can change the default log-level. That commit changed the way to determine whether current app is running in spark-shell or not but it left the log-level part as it is, which causes this regression. SPARK-25118 changed the default log level to INFO for spark-shell because of which the progress bar is not enabled anymore. This commit will remove the log-level check for enabling progress bar for spark-shell as it is not necessary and seems to be a leftover from SPARK-21568 Testing Done: 1. Ensured that progress bar is enabled with spark-shell by default 2. Ensured that progress bar is not enabled with spark-submit
|
@vanzin @dongjoon-hyun since you worked on #19061 |
|
Test build #101562 has finished for PR 23618 at commit
|
vanzin
reviewed
Jan 23, 2019
core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
Outdated
Show resolved
Hide resolved
|
Test build #101598 has finished for PR 23618 at commit
|
|
Test build #101657 has finished for PR 23618 at commit
|
|
Merging to master. |
jackylee-ch
pushed a commit
to jackylee-ch/spark
that referenced
this pull request
Feb 18, 2019
…rk-shell ## What changes were proposed in this pull request? SPARK-21568 made a change to ensure that progress bar is enabled for spark-shell by default but not for other apps. Before that change, this was distinguished using log-level which is not a good way to determine the same as users can change the default log-level. That commit changed the way to determine whether current app is running in spark-shell or not but it left the log-level part as it is, which causes this regression. SPARK-25118 changed the default log level to INFO for spark-shell because of which the progress bar is not enabled anymore. This commit will remove the log-level check for enabling progress bar for spark-shell as it is not necessary and seems to be a leftover from SPARK-21568 ## How was this patch tested? 1. Ensured that progress bar is enabled with spark-shell by default 2. Ensured that progress bar is not enabled with spark-submit Closes apache#23618 from ankuriitg/ankurgupta/SPARK-26694. Authored-by: ankurgupta <ankur.gupta@cloudera.com> Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
SPARK-21568 made a change to ensure that progress bar is enabled for spark-shell
by default but not for other apps. Before that change, this was distinguished
using log-level which is not a good way to determine the same as users can change
the default log-level. That commit changed the way to determine whether current
app is running in spark-shell or not but it left the log-level part as it is,
which causes this regression. SPARK-25118 changed the default log level to INFO
for spark-shell because of which the progress bar is not enabled anymore.
This commit will remove the log-level check for enabling progress bar for
spark-shell as it is not necessary and seems to be a leftover from SPARK-21568
How was this patch tested?