IPython3 error:'interp' is not a recognized option

[Anne623]

Hi, I use ISpark in IPython3.0, the kernel.json is below:

{
    "display_name": "ISpark",
    "language": "scala",
    "argv": [
              "/opt/spark/spark-1.2.1/bin/spark-submit",
                "--master",
                "spark://x.xx.xxx.xxx:7077",
                "--total-executor-cores","3",
                "--class",
                "org.tribbloid.ispark.Main",
                "--executor-memory", "2G",
                "/pathto/ispark-core-assembly-0.2.0-SNAPSHOT.jar",
                "--profile",
                "{connection_file}",
                "--interp",
                "Spark",
                "--parent"
     ],
     "codemirror_mode": "scala"
}

while i use ISpark,it's logs:

Spark assembly has been built with Hive, including Datanucleus jars on classpath
Exception in thread "main" joptsimple.UnrecognizedOptionException: 'interp' is not a recognized option
    at joptsimple.OptionException.unrecognizedOption(OptionException.java:89)
    at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:429)
    at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
    at joptsimple.OptionParser.parse(OptionParser.java:361)
    at org.tribbloid.ispark.Options.<init>(Options.scala:12)
    at org.tribbloid.ispark.Main$.main(Main.scala:192)
    at org.tribbloid.ispark.Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Dose my kerner.json is wrong?

[tribbloid]

Sorry for the delay in documentation, --interp option has been removed as different main classes can be submitted for different extensions (which makes it redundant. I'll post a recommended profile shortly)

[tribbloid]

Plz check the updated example config file in readme:

import os
c = get_config()

SPARK_HOME = os.environ['SPARK_HOME']
# the above line can be replaced with: SPARK_HOME = '${INSERT_INSTALLATION_DIR_OF_SPARK}'
MASTER = '${INSERT_YOUR_SPARK_MASTER_URL}'

c.KernelManager.kernel_cmd = [SPARK_HOME+"/bin/spark-submit",
 "--master", MASTER,
 "--class", "org.tribbloid.ispark.Main",
 "--executor-memory", "2G",
#(only enable this line if you have extra jars) "--jars", "${FULL_PATHS_OF_EXTRA_JARS}",
 "${FULL_PATH_OF_MAIN_JAR}",
 "--profile", "{connection_file}",
 "--parent"]

c.NotebookApp.ip = '*' # only add this line if you want IPython-notebook being open to the public
c.NotebookApp.open_browser = False # only add this line if you want to suppress opening a browser after IPython-notebook initialization
c.NotebookApp.port = 8888

[Anne623]

Ok, Thank you,:)
I want to know how config kernel_spec file in IPython3.0 Released.

[tribbloid]

Can you try the following spec first?

{
    "display_name": "ISpark",
    "language": "scala",
    "argv": [
              "/opt/spark/spark-1.2.1/bin/spark-submit",
                "--master",
                "local[*]",
                "--class",
                "org.tribbloid.ispark.Main",
                "--executor-memory", "2G",
                "/pathto/ispark-core-assembly-0.2.0-SNAPSHOT.jar",
                "--profile",
                "{connection_file}",
                "--parent"
     ],
     "codemirror_mode": "scala"
}

I tried IPy 3.0 before it's release, it works in most of the time but occasionally the kernel is out of synch. I'll try it again some da

[tribbloid]

Hi @Anne623, how is it working? Is it running smoothly?

[Anne623]

Hi, @tribbloid , I have not check yet, I will feed back while I checked.

[Anne623]

Hi, @tribbloid , I checked, it's working. Thank you. 👍 :)

[tribbloid]

@Anne623, Thanks a lot for verification! It really helps a lot of people about to convert to IPy3! (like me)