Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for cell-level sparkmagic language type #509

Open
Tagar opened this issue Jan 28, 2019 · 1 comment
Open

add support for cell-level sparkmagic language type #509

Tagar opened this issue Jan 28, 2019 · 1 comment
Labels
kind:enhancement A new Sparkmagic feature

Comments

@Tagar
Copy link

Tagar commented Jan 28, 2019

FYI - Livy 0.5+ has session kind which is actually useless

See this comment in LIVY-469

Starting from 0.5 session kind is actually useless, we should encourage user to not specify session kind, instead we should set code kind when submitting statement.

So this could be both a notebook-level property (just as a default for newly created cells), and a cell-level property.
For example, this opens up a way to for something like this in sparkmagic:

%%spark -l python
# .. some pyspark code.. 
df = spark.sql("select * from universe")
%%spark -l scala 
// .. some Scala code.. 
var df = spark.sql("select * from universe")

on the same (shared) backend Spark session.

@Tagar
Copy link
Author

Tagar commented Jan 28, 2019

Spin-off from #507

@apetresc apetresc added the kind:enhancement A new Sparkmagic feature label Jan 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:enhancement A new Sparkmagic feature
Projects
None yet
Development

No branches or pull requests

2 participants