Skip to content

Commit

Permalink
Update website
Browse files Browse the repository at this point in the history
  • Loading branch information
Bo Peng committed Aug 3, 2017
1 parent 66c0334 commit 9296a1d
Show file tree
Hide file tree
Showing 12 changed files with 12,701 additions and 11,603 deletions.
25 changes: 22 additions & 3 deletions docs/Documentation.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,6 @@ <h1 id="SoS-Reference">SoS Reference<a class="anchor-link" href="#SoS-Reference"
<li><p><a href="doc/documentation/SoS_Syntax.html">SoS syntax and structure of SoS scripts</a><br>
<small>An overview of the SoS syntax and key components of SoS scripts</small></p>
</li>
<li><p><a href="doc/documentation/Notebook_Interface.html">SoS Jupyter kernel, features and magics</a><br>
<small>Using SoS with Jupyter notebook</small></p>
</li>
<li><p><a href="doc/documentation/SoS_Step.html">Step options, statements and their options</a><br>
<small>Section options, input, output, depends statements and their options.</small></p>
</li>
Expand All @@ -50,6 +47,28 @@ <h1 id="SoS-Reference">SoS Reference<a class="anchor-link" href="#SoS-Reference"
</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="SoS-Notebook">SoS Notebook<a class="anchor-link" href="#SoS-Notebook">&#182;</a></h1><ul>
<li><p><a href="doc/documentation/Notebook_Interface.html">SoS Jupyter kernel, features and magics</a><br>
<small>Using SoS with Jupyter notebook</small></p>
</li>
<li><p><a href="doc/documentation/SoS_Magics.html">SoS Magics</a><br>
<small>Reference to all SoS Magics</small></p>
</li>
<li><p><a href="doc/documentation/Supported_Languages.html">Supported languages</a><br>
<small>Data exchange between SoS and supported languages</small></p>
</li>
<li><p><a href="doc/documentation/Language_Module.html">Adding a new language</a><br>
<small>Allow data exchange between SoS and a new langauge.</small></p>
</li>
</ul>

</div>
</div>
</div>
Expand Down
142 changes: 2 additions & 140 deletions docs/doc/documentation/Extending_SoS.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@
});
</script><style> /* defined here in case the main.css below cannot be loaded */

.lan_Bash .input_prompt { background-color: #E6EEFF !important }.lan_JavaScript .input_prompt { background-color: #00ff80 !important }.lan_Python2 .input_prompt { background-color: #F6FAEA !important }.lan_rik_local_remote {}.lan_R .input_prompt { background-color: #FDEDEC !important }.lan_Python3 .input_prompt { background-color: #EAFAF1 !important }.lan_SoS {}
.lan_Bash .input_prompt { background-color: #E6EEFF !important }.lan_JavaScript .input_prompt { background-color: #00ff80 !important }.lan_Python2 .input_prompt { background-color: #EAFAF1 !important }.lan_rik_local_remote {}.lan_R .input_prompt { background-color: #FDEDEC !important }.lan_Python3 .input_prompt { background-color: #EAFAF1 !important }.lan_SoS {}

</style>
<body>
Expand All @@ -229,7 +229,7 @@ <h1 id="Extending-SoS">Extending SoS<a class="anchor-link" href="#Extending-SoS"
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>SoS can be easily extended with new actions, targets, converters, file previewers, and support for new languages. To make the extension available to other users, you can either create and distribute a separate package, or extend SoS and send us a <a href="https://help.github.com/articles/about-pull-requests/">pull request</a>.</p>
<p>SoS can be easily extended with new actions, targets, converters, file previewers. To make the extension available to other users, you can either create and distribute a separate package, or extend SoS and send us a <a href="https://help.github.com/articles/about-pull-requests/">pull request</a>.</p>

</div>
</div>
Expand Down Expand Up @@ -410,144 +410,6 @@ <h2 id="Preview-additional-formats">Preview additional formats<a class="anchor-l
</li>
</ol>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Support-for-a-language.">Support for a language.<a class="anchor-link" href="#Support-for-a-language.">&#182;</a></h2><p>SoS needs to know a few things before it can support a language properly,</p>
<ol>
<li>The Jupyter kernel this language uses to work with Jupyer, which is a <code>ir</code> kernel for language <code>R</code>.</li>
<li>How to translate a Python object to a <strong>similar</strong> object in this language</li>
<li>How to translate an object in this language to a <strong>similar</strong> object in Python.</li>
<li>The color of the prompt of cells executed by this language.</li>
<li>(Optional but recommend). Information of a running session.</li>
<li>Optional options for interacting with the language on frontend.</li>
</ol>
<p>It is important to understand that, instead of providing object that is native to the <strong>sender</strong> language (e.g. use <a href="http://rpy2.bitbucket.org/"><code>rpy2</code></a> to wrap R objects in Python), SoS tries to provide object that is native to the <strong>recipient</strong> language. That is to say, although objects</p>

<pre><code>a = 1
b = c(1, 2)</code></pre>
<p>are of the same type <code>numeric</code> in R, they are translated to Python in different types</p>

<pre><code>a = 1
b = [1, 2]</code></pre>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>To support a new language, you will need to write a Python package that defines a class, say <code>mylanguage</code>, that should provide the following class attributes:</p>
<ol>
<li><code>supported_kernels</code>: a dictionary of language and names of the kernels that the language supports, such as <code>{'R': ['ir']}</code>. If multiple kernels are supported, SoS will look for a kernel with matched name in the order that is specified (e.g. <code>{'JavaScript': ['ijavascript', 'inodejs']}</code>). Multiple languages can be specified if a language module supports multiple languages (e.g. <code>Matlab</code> and <code>Octave</code>).</li>
<li><code>background_color</code>: a name or <code>#XXXXXX</code> value for a color that will be used in the prompt area of cells that are executed by the subkernel. An empty string can be used for using default notebook color.</li>
<li><code>options</code>: A Python dictionary with options that will be passed to the frontend. Currently two options <code>variable_pattern</code> and <code>assignment_pattern</code> are supported. Both options should be regular expressions in JS style. <ul>
<li>Option <code>variable_pattern</code> is used to identify if a statement is a simple variable (nothing else). If this option is defined and the input text (if executed at the side panel) matches the pattern, SoS will prepend <code>%preview</code> to the code. This option is useful only when <code>%preview var</code> displays more information than <code>var</code>.</li>
<li>Option <code>assignment_pattern</code> is used to identify if a statement is an assignment operation. If this option is defined and the input text matches the pattern, SoS will prepend <code>%preview var</code> to the code where <code>var</code> should be the first matched portion of the pattern (use <code>( )</code>). This mechanism allows SoS to automatically display result of an assignment when you step through the code.</li>
</ul>
</li>
</ol>
<p>An instance of the class would be initialized with the sos kernel and the name of the subkernel, which does not have to be one of the <code>supported_kernels</code> (could be self-defined) and should provide the following attributes and functions:</p>
<ol>
<li><code>init_statements</code>: a statement that will be executed by the sub-kernel when the kernel starts. This statement usually defines functions to convert object to Python.</li>
<li><code>get_vars</code>: a Python function that transfer a Python variable to the subkernel.</li>
<li><code>put_vars</code>: a Python function that put one or more variables in the subkernel to SoS or another subkernel.</li>
<li><code>sessioninfo</code>: a Python function that returns information of the running kernel, usually including version of the language, the kernel, and currently used packages and their versions. For <code>R</code>, this means a call to <code>sessionInfo()</code> function. The return value of this function can be a string, a list of strings or <code>(key, value)</code> pairs, or a dictinary. The function will be called by the <code>%sessioninfo</code> magic of SoS.</li>
</ol>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Obtain-variable-from-SoS">Obtain variable from SoS<a class="anchor-link" href="#Obtain-variable-from-SoS">&#182;</a></h3><p>The <code>get_vars</code> function should be defined as</p>

<pre><code>def get_vars(self, var_names)</code></pre>
<p>where</p>
<ul>
<li><code>self</code> is the language instance with access to the SoS kernel, and</li>
<li><code>var_names</code> are names in the sos dictionary.</li>
</ul>
<p>This function is responsible for probing the type of Python variable and create a similar object in the subkernel.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For example, to create a Python object <code>b = [1, 2]</code> in <code>R</code> (magic <code>%get</code>), this function could</p>
<ol>
<li>Obtain a R expression to create this variable (e.g. <code>b &lt;- c(1, 2)</code>)</li>
<li>Execute the expression in the subkernel to create variable <code>b</code> in it.</li>
</ol>
<p>Note that the function <code>get_vars</code> can change the variable name because a valid variable name in Python might not be a valid variable name in another language. The function should give a warning if this happens.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Send-variables-to-other-kernels">Send variables to other kernels<a class="anchor-link" href="#Send-variables-to-other-kernels">&#182;</a></h3><p>The <code>put_vars</code> function should be defined as</p>

<pre><code>def put_vars(self, var_names, to_kernel=None)</code></pre>
<p>where</p>
<ol>
<li><code>self</code> is the language instance with access to the SoS kernel</li>
<li><code>var_name</code> is a list of variables that should exist in the subkernel. Because a subkernel is responsible for sharing variables with names starting with <code>sos</code> to SoS automatically, this function should be called to pass these variables even when <code>var_names</code> is empty.</li>
<li><code>to_kernel</code> is the destination kernel to which the variables should be passed.</li>
</ol>
<p>Depending on destination kernel, this function can:</p>
<ul>
<li>If the destination kernel is <code>sos</code>, the function should return a dictionary of variables that will be merged to the SoS dictionary.</li>
<li>If direct variable transfer is not supported by the language, the function can return a Python dictionary, in which case the language transfers the variables to SoS and let SoS pass along to the destination kernel.</li>
<li>If direct variable transfer is supported, the function should return a string. SoS will evaluate the string in the destination kernel to pass variables directly to the destination kernel.</li>
</ul>
<p>So basically, a language can start with an implementation of <code>put_vars(to_kernel='sos')</code> and let SoS handle the rest. If need arises, it can</p>
<ul>
<li>Implement variable exchanges between instances of the same language. This can be useful because there are usually lossness and more efficient methods in this case.</li>
<li>Put variable to another languages where direct varable transfer is much more efficient than transferring through SoS.</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For example, to send a <code>R</code> object <code>b &lt;- c(1, 2)</code> from subkernel <code>R</code> to <code>SoS</code> (magic <code>%put</code>), this function can</p>
<ol>
<li>Execute an statement in the subkernel to get the value(s) of variable(s) in some format, for example, a string <code>"{'b': [1, 2]}"</code>.</li>
<li>Post-process these varibles to return a dictionary to SoS.</li>
</ol>
<p>The <a href="https://github.com/vatlab/SOS/blob/master/src/sos/R/kernel.py"><code>R</code> sos extension</a> provides a good example to get you started.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><strong>NOTE</strong>: Unlike other language extension mechanisms in which the python module can get hold of the "engine" of the interpreter (e.g. <code>saspy</code> and matlab's Python extension start the interpreter for direct communication) or have access to lower level API of the language (e.g. <code>rpy2</code>), SoS only have access to the interface of the language and perform all conversions by executing commands in the subkernels and intercepting their response. Consequently,</p>
<ol>
<li>Data exchange can be slower than other methods.</li>
<li>Data exchange is less dependent on version of the interpreter.</li>
<li>Data exchange can happen between a local and a remote kernel.</li>
</ol>
<p>Also, although it can be more efficient to save large datasets to disk files and load in another kernel, this method does not work for kernels that do not share the same filesystem. We currently ignore this issue and assume all kernels have access to the same file system.</p>

</div>
</div>
</div>
Expand Down
Loading

0 comments on commit 9296a1d

Please sign in to comment.