Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-4534][Core]JavaSparkContext create new constructor to support preferredNodeLocalityData with YARN #3403

Closed
wants to merge 1 commit into from

Conversation

lianhuiwang
Copy link
Contributor

create new constructor to support preferredNodeLocalityData with YARN
example:
SparkConf sparkConf=new SparkConf();
Configuration conf = new Configuration();
List inputFormatInfoList =new ArrayList();
inputFormatInfoList.add(new InputFormatInfo(conf, org.apache.hadoop.mapred.TextInputFormat.class, path));
JavaSparkContext ctx=new JavaSparkContext(sparkConf,inputFormatInfoList);

@SparkQA
Copy link

SparkQA commented Nov 21, 2014

Test build #23724 has started for PR 3403 at commit ff93679.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Nov 21, 2014

Test build #23724 has finished for PR 3403 at commit ff93679.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23724/
Test FAILed.

@sryza
Copy link
Contributor

sryza commented Nov 21, 2014

preferredNodeLocalityData is currently broken (see SPARK-2089), and we're discussing changing the API for it. I think it would be best to hold off on this change until we've made a decision there.

@tgravescs
Copy link
Contributor

I agree with Sandy on this, can you close this until we get SPARK-2089 working and then we can make sure it works with Java api also.

@lianhuiwang
Copy link
Contributor Author

OK,i will close this。thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants