Support running on yarn cluster

[lev112]

Hi,

are there plans to support yarn cluster?

[tribbloid]

It should be running smoothly without extra effort (just change the
ipython profile to submit it to yarn), have you tried it?

Yours Peng

On 02/13/2015 12:57 AM, Lev wrote:

Hi,

are there plans to support yarn cluster?

—
Reply to this email directly or view it on GitHub
#9.

[lev112]

I did,
and I managed to run it,
but it would fail after being submitted to yarn.

I will recreate this and post the error I got

Thanks,
Lev.

[meniluca]

Hi all, I'm really interested about this issue. I would like to be able to do what I'm doing now with "spark-submit --master yarn-cluster" with ISpark. Is there any possibility to configure ISpark to submit operation in cluster mode?
Cheers,
Luca

[tribbloid]

Sorry for the late reply.
The original Spark-shell and all derivative projects (including Zeppelin) cannot be run in cluster mode. Client mode is the only option (if you try you'll get an error message, furthermore, how could you interact with a driver that is randomly dispatched?)

[meniluca]

Thank you very much for the answer!

[tribbloid]

You are welcome. This will be marked as 'solved', I'm still not sure if running on YARN is bug-free, but please open a new ticket in that case.