Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-13626] [core] Avoid duplicate config deprecation warnings. #11510

Closed
wants to merge 4 commits into from
Closed

[SPARK-13626] [core] Avoid duplicate config deprecation warnings. #11510

wants to merge 4 commits into from

Commits on Mar 4, 2016

  1. [SPARK-13626] [core] Avoid duplicate config deprecation warnings. 

    Three different things were needed to get rid of spurious warnings:
- silence deprecation warnings when cloning configuration
- change the way SparkHadoopUtil instantiates SparkConf to silence
  warnings
- avoid creating new SparkConf instances where it's not needed.

On top of that, I changed the way that Logging.scala detects the repl;
now it uses a method that is overridden in the repl's Main class, and
the hack in Utils.scala is not needed anymore. This makes the 2.11 repl
behave like the 2.10 one and set the default log level to WARN, which
is a lot better. Previously, this wasn't working because the 2.11 repl
triggers log initialization earlier than the 2.10 one.

I also removed and simplified some other code in the 2.11 repl's Main
to avoid replicating logic that already exists elsewhere in Spark.

Last but not least, fixed a compilation bug in a test for Scala 2.10.

Tested the 2.11 repl in local and yarn modes.
    Marcelo Vanzin committed Mar 4, 2016
    Configuration menu
    Copy the full SHA
    c5338f6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into SPARK-13626

    Marcelo Vanzin committed Mar 4, 2016
    Configuration menu
    Copy the full SHA
    c480c27 View commit details
    Browse the repository at this point in the history

  3. Slightly terser and more explicit log initialization for the shell.

    Marcelo Vanzin committed Mar 4, 2016
    Configuration menu
    Copy the full SHA
    d01d9ba View commit details
    Browse the repository at this point in the history

  4. Fix ReplSuite.

    Marcelo Vanzin committed Mar 4, 2016
    Configuration menu
    Copy the full SHA
    d7f1ffa View commit details
    Browse the repository at this point in the history