-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-2889] Create Hadoop config objects consistently. #1843
Commits on Aug 7, 2014
-
Replace explicit Configuration instantiation with SparkHadoopUtil.
This is the basic grunt work; code doesn't fully compile yet, since I'll do some of the more questionable changes in separate commits.
Marcelo Vanzin committedAug 7, 2014 Configuration menu - View commit details
-
Copy full SHA for 1e7003f - Browse repository at this point
Copy the full SHA 1e7003fView commit details -
Update Utils API to take a Configuration argument.
Instead of using "new Configuration()" where a configuration is needed, let the caller provide a context-appropriate config object.
Marcelo Vanzin committedAug 7, 2014 Configuration menu - View commit details
-
Copy full SHA for b8ab173 - Browse repository at this point
Copy the full SHA b8ab173View commit details -
Initialize config in SparkHadoopUtil.
This is sort of hackish, since it doesn't account for any customization someone might make to SparkConf before they actually start executing spark code. Instead, this will only consider options available in the system properties when creating the hadoop conf.
Marcelo Vanzin committedAug 7, 2014 Configuration menu - View commit details
-
Copy full SHA for f16cadd - Browse repository at this point
Copy the full SHA f16caddView commit details -
Marcelo Vanzin committed
Aug 7, 2014 Configuration menu - View commit details
-
Copy full SHA for 3f26760 - Browse repository at this point
Copy the full SHA 3f26760View commit details
Commits on Aug 20, 2014
-
Merge branch 'master' into SPARK-2889
Conflicts: core/src/main/scala/org/apache/spark/util/FileLogger.scala yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
Marcelo Vanzin committedAug 20, 2014 Configuration menu - View commit details
-
Copy full SHA for 0ac3fdf - Browse repository at this point
Copy the full SHA 0ac3fdfView commit details
Commits on Aug 26, 2014
-
Merge branch 'master' into SPARK-2889
Conflicts: core/src/main/scala/org/apache/spark/scheduler/cluster/SimrSchedulerBackend.scala
Marcelo Vanzin committedAug 26, 2014 Configuration menu - View commit details
-
Copy full SHA for fc45067 - Browse repository at this point
Copy the full SHA fc45067View commit details -
Restore old method for backwards compat.
Marcelo Vanzin committedAug 26, 2014 Configuration menu - View commit details
-
Copy full SHA for 3d345cb - Browse repository at this point
Copy the full SHA 3d345cbView commit details
Commits on Aug 27, 2014
-
Marcelo Vanzin committed
Aug 27, 2014 Configuration menu - View commit details
-
Copy full SHA for 53f9506 - Browse repository at this point
Copy the full SHA 53f9506View commit details -
Add test to ensure that overriding Yarn configs works.
Marcelo Vanzin committedAug 27, 2014 Configuration menu - View commit details
-
Copy full SHA for 51e71cf - Browse repository at this point
Copy the full SHA 51e71cfView commit details -
Merge branch 'master' into SPARK-2889
Conflicts: yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/ExecutorLauncher.scala yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientClusterScheduler.scala yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterScheduler.scala yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ExecutorLauncher.scala
Marcelo Vanzin committedAug 27, 2014 Configuration menu - View commit details
-
Copy full SHA for f179013 - Browse repository at this point
Copy the full SHA f179013View commit details
Commits on Aug 29, 2014
-
Merge branch 'master' into SPARK-2889
Conflicts: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala
Marcelo Vanzin committedAug 29, 2014 Configuration menu - View commit details
-
Copy full SHA for 52daf35 - Browse repository at this point
Copy the full SHA 52daf35View commit details