- All prerequisites listed in Build in Windows
- .NET Framework 4.6.1 or above.
To Be Noticed: Currently Mobius Shell only supports Windows, Linux support is not supported yet.
Mobius Shell is included in Mobius release, please refer to Build in Windows to build Mobius.
Please first refer to Running Mobius App to make sure all required softwares and enviroment variables are placed or set properly.
- Local
scripts\sparkclr-repl.cmd --conf spark.local.dir=D:\tmp\repl
- Yarn
scripts\sparkclr-repl.cmd --master yarn-client --num-executors 2 --executor-cores 2 --executor-memory 2G
For more information about supported options with different deploy modes, please refer to sparkclr-shell.cmd --help
| Directive | Description |
|---|---|
:help |
Display help on available commands |
:load |
Load extra local library to current execution context, e.g. :load "myLib.dll" |
:quit |
Exit Mobius Shell |
scripts\sparkclr-shell.cmd
Spark context available as sc.
SQL context available as sqlContext.
Use :quit to exit.
Type ":help" for more information.
>
> :help
Commands:
:help Display help on available commands.
:load Load extra library to current execution context, e.g. :load "myLib.dll".
:quit Exit REPL.
>
> // example of loading extra library
> :load "d:\test\TestLib.dll"
Loaded assebmly from d:\test\TestLib.dll
> using TestLib;
> new Lib().Test()
This is from test lib.
>
> // RDD example
> using System.Linq;
> var rdd = sc.Parallelize(Enumerable.Range(0, 100), 2).Map(d => d+1);
> rdd.Count()
100
>
> // dataframe example
> var df = sqlContext.Read().Json(@"D:\SparkWorkspace\SparkCLR_REPL\build\runtime\data\people.json");
> df.Count()
4
> df.Show()
+--------------------+---+---+-----+
| address|age| id| name|
+--------------------+---+---+-----+
| [Columbus,Ohio]| 34|123| Bill|
| [null,California]| 44|456|Steve|
|[Seattle,Washington]| 43|789| Aaron|
+--------------------+---+---+-----+
> :quit