1.1.4

CONTRIBUTING.md

@@ -0,0 +1,73 @@
+# Contributing to Jupytext
+
+Thanks for reading this! Contributions to this project are welcome.
+There are many ways you can contribute...
+
+## Spread the word
+
+You like Jupytext? Probably your friends and colleagues will like it too. 
+Show them what you've been able to do with: version control, collaboration on notebooks, refactoring of notebooks, notebooks integrated in library, notebook generated from Markdown documents...
+
+By the way, we're also interested to know how you use Jupytext! There may well be applications we've not thought of!
+
+## Improve the documentation
+
+You think the documentation could be improved? You've spotted a typo, or you think you can rephrase a paragraph to make is easier to follow? Please follow the _Edit on Github_ link, edit the document, and submit a pull request!
+
+## Report an issue
+
+You have seen an issue with Jupytext, or you can't find your way in the [documentation](https://jupytext.readthedocs.io)?
+Please let us know, and provide enough information so that we can reproduce the problem.
+
+## Propose enhancements
+
+You want to submit an enhancement on Jupytext? Unless this is a small change, we usually prefer that you let us know beforehand: open an issue that describe the problem you want to solve.
+
+A pull request for which you do not need to contact us in advance is the addition of a new language to Jupytext. In principle that should be easy - you would only have to:
+- document the language extension and comment by adding one line to `_SCRIPT_EXTENSIONS` in `languages.py`.
+- contribute a sample notebook in `tests/notebooks/ipynb_[language]`.
+- add two tests in `test_mirror.py`: one for the `light` format, and another one for the `percent` format.
+- Make sure that the tests pass, and that the text representations of your notebook, found in  `tests/notebooks/mirror/ipynb_to_script` and `tests/notebooks/mirror/ipynb_to_percent`, are valid scripts.
+
+# How to setup a development environment for Jupytext
+
+## Jupytext as a Python package
+
+Most of Jupytext's code is written in Python. To develop the Python part of Jupytext, you should clone Jupytext, then create a dedicated Python env:
+```
+cd jupytext
+conda create -n jupytext-dev python=3.6 notebook mock pyyaml
+conda activate jupytext-dev
+pip install -r requirements*.txt
+```
+
+Tests are executed with `pytest` (install `pytest-xdist` and then run `pytest -n 3` if you want them to run in parallel). If you develop `jupytext` command line, you can install Jupytext in dev mode with
+```
+pip install -e .
+````
+If you also need the Jupytext extensions for Jupyter, then it is better to build the package in full, and install it with
+```
+python setup.py sdist bdist_wheel
+pip install dist/jupytext-XXX.tar.gz
+```
+
+## Jupytext's extension for Jupyter Notebook
+
+Our extension for Jupyter Notebook adds a Jupytext entry to Jupyter Notebook. The code is found at `jupytext/nbextension/index.js`. Instructions to develop that extension are found [here](https://github.com/mwouts/jupytext_nbextension).
+
+## Jupytext's extension for JupyterLab
+
+Our extension for JupyterLab adds a series of Jupytext commands to JupyterLab. The code is in `packages/labextension`. See the `README.md` there for instructions on how to develop that extension.
+
+## Jupytext's documentation
+
+Install the documentation tools with
+```
+conda activate jupytext-dev
+cd docs
+pip install -r doc-requirements.txt
+```
+and build the HTML documentation locally with
+```
+make html
+```

HISTORY.rst

@@ -3,6 +3,23 @@
 Release History
 ---------------
 
+1.1.4 (2019-06-05)
+++++++++++++++++++++++
+
+**Improvements**
+
+- New argument ``--set-kernel`` in Jupytext command line (#230)
+- Jupytext now accepts ``--to script`` or ``--to auto`` (#240)
+- Jupytext now has a real Sphinx documentation on `readthedocs
+<https://jupytext.readthedocs.io/en/latest/>`_, thanks to Chris Holdgraf (#237)
+
+**BugFixes**
+
+- Invalid notebooks may cause a warning, but not a fatal error (#234)
+- Jupyter server extension leaves the contents manager unchanged if it is a sub-class of Jupytext's CM (#236)
+- Fixed format inference when metadata is present but not format information (#239)
+- Preserve executable and encoding information in scripts with metadata (#241)
+
 1.1.3 (2019-05-22)
 ++++++++++++++++++++++
 

README.md

@@ -1,6 +1,7 @@
 # Jupyter notebooks as Markdown documents, Julia, Python or R scripts
 
 [![Build Status](https://travis-ci.com/mwouts/jupytext.svg?branch=master)](https://travis-ci.com/mwouts/jupytext)
+[![Documentation Status](https://readthedocs.org/projects/jupytext/badge/?version=latest)](https://jupytext.readthedocs.io/en/latest/?badge=latest)
 [![codecov.io](https://codecov.io/github/mwouts/jupytext/coverage.svg?branch=master)](https://codecov.io/github/mwouts/jupytext?branch=master)
 [![Language grade: Python](https://img.shields.io/badge/lgtm-A+-brightgreen.svg)](https://lgtm.com/projects/g/mwouts/jupytext/context:python)
 
@@ -115,4 +116,4 @@ For more examples, see the [jupytext documentation](https://jupytext.readthedocs
 
 ## Want to contribute?
 
-Contributions are welcome. Please let us know how you use `jupytext` and how we could improve it. You think the documentation could be improved? Go ahead! And stay tuned for more demos on [medium](https://medium.com/@marc.wouts) and [twitter](https://twitter.com/marcwouts)!
+Contributions are welcome. Please let us know how you use `jupytext` and how we could improve it. You think the documentation could be improved? Go ahead! Read our [`CONTRIBUTING.md`](CONTRIBUTING.md) to find out guidelines and instructions on how to setup a development environment. And stay tuned for more demos on [medium](https://medium.com/@marc.wouts) and [twitter](https://twitter.com/marcwouts)!

docs/examples.md

@@ -41,7 +41,5 @@ In the animation below we propose a quick demo of Jupytext. While the example re
 Jupytext allows to import code from other Jupyter notebooks in a very simple manner. Indeed, all you need to do is to pair the notebook that you wish to import with a script, and import the resulting script.
 
 If the notebook contains demos and plots that you don't want to import, mark those cells as either
-- _active_ only in the `ipynb` format, with the `{"active": "ipynb"}` cell metadata, or with an `active-ipynb` tag (you may use the [jupyterlab-celltags](https://github.com/jupyterlab/jupyterlab-celltags) extension for this).
+- _active_ only in the `ipynb` format, with the `{"active": "ipynb"}` cell metadata, or with an `active-ipynb` tag (you may use the [jupyterlab-celltags](https://github.com/jupyterlab/jupyterlab-celltags) extension for this). Use a `{"active": "ipynb,py"}` metadata or a an `active-ipynb-py` tag if you want the cell to be active only in the `ipynb` and `py` formats, but not in the R Markdown format.
 - _frozen_, using the [freeze extension](https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/freeze/readme.html) for Jupyter notebook.
-
-Inactive cells will be commented in the paired script, and consequently will not be executed when the script is imported.

docs/index.md

@@ -8,4 +8,3 @@
 * [Using Jupytext in Jupyter](using-server)
 * [Using at the command line](using-cli)
 * [Supported document formats](formats)
-* [Advanced configuration and use](advanced)

jupytext/__init__.py

@@ -17,7 +17,7 @@ def __init__(self, parent=None, log=None):  # noqa
 
 def load_jupyter_server_extension(app):  # pragma: no cover
     """Use Jupytext's contents manager"""
-    if isinstance(app.contents_manager_class, TextFileContentsManager):
+    if issubclass(app.contents_manager_class, TextFileContentsManager):
         app.log.info("[Jupytext Server Extension] NotebookApp.contents_manager_class is "
                      "(a subclass of) jupytext.TextFileContentsManager already - OK")
         return

jupytext/cli.py

@@ -8,12 +8,15 @@
 import argparse
 import json
 from copy import copy
+from jupyter_client.kernelspec import find_kernel_specs, get_kernel_spec
 from .jupytext import readf, reads, writef, writes
 from .formats import _VALID_FORMAT_OPTIONS, _BINARY_FORMAT_OPTIONS, check_file_version
-from .formats import long_form_one_format, long_form_multiple_formats, short_form_one_format
+from .formats import long_form_one_format, long_form_multiple_formats, short_form_one_format, auto_ext_from_metadata
+from .header import recursive_update
 from .paired_paths import paired_paths, base_path, full_path, InconsistentPath
 from .combine import combine_inputs_with_outputs
 from .compare import test_round_trip_conversion, NotebookDifference
+from .kernels import kernelspec_from_language
 from .version import __version__
 
 
@@ -59,8 +62,8 @@ def parse_jupytext_args(args=None):
                              'in the git index, which have an extension that matches the (optional) --from argument.')
     # Destination format & act on metadata
     parser.add_argument('--to',
-                        help="Destination format: either one of 'notebook', 'markdown', 'rmarkdown', any valid "
-                             "notebook extension, or a full format '[prefix_path//][suffix.]ext[:format_name]")
+                        help="Destination format: either one of 'notebook', 'markdown', 'rmarkdown', 'script', any "
+                             "valid notebook extension, or a full format '[prefix_path//][suffix.]ext[:format_name]")
     parser.add_argument('--format-options', '--opt',
                         action='append',
                         help='Set format options with e.g. --opt comment_magics=true '
@@ -69,6 +72,12 @@ def parse_jupytext_args(args=None):
                         type=str,
                         help='Set jupytext.formats metadata to the given value. Use this to activate pairing on a '
                              'notebook, with e.g. --set-formats ipynb,py:light')
+    parser.add_argument('--set-kernel', '-k',
+                        type=str,
+                        help="Set the kernel with the given name on the notebook. Use '--set-kernel -' to set "
+                             "a kernel matching the current environment on Python notebooks, and matching the "
+                             "notebook language otherwise "
+                             "(get the list of available kernels with 'jupyter kernelspec list')")
     parser.add_argument('--update-metadata',
                         default={},
                         type=json.loads,
@@ -86,7 +95,7 @@ def parse_jupytext_args(args=None):
 
     # Action: convert(default)/version/list paired paths/sync/apply/test
     action = parser.add_mutually_exclusive_group()
-    action.add_argument('--version',
+    action.add_argument('--version', '-v',
                         action='store_true',
                         help="Show jupytext's version number and exit")
     action.add_argument('--paired-paths', '-p',
@@ -188,7 +197,7 @@ def writef_git_add(notebook_, nb_file_, fmt_):
     if not args.to and not args.output and not args.sync \
             and not args.pipe and not args.check \
             and not args.test and not args.test_strict \
-            and not args.update_metadata:
+            and not args.update_metadata and not args.set_kernel:
         raise ValueError('Please select an action')
 
     if args.output and len(args.notebooks) != 1:
@@ -244,15 +253,52 @@ def writef_git_add(notebook_, nb_file_, fmt_):
             elif ext:
                 fmt = {'extension': ext}
 
+        # Compute actual extension when using script/auto, and update nb_dest if necessary
+        dest_fmt = args.to
+        if dest_fmt and dest_fmt['extension'] == '.auto':
+            auto_ext = auto_ext_from_metadata(notebook.metadata)
+            if not auto_ext:
+                raise ValueError('The notebook has no language information. '
+                                 'Please provide an explicit script extension.')
+            dest_fmt['extension'] = auto_ext
+            if not args.output and nb_file != '-':
+                nb_dest = full_path(base_path(nb_file, args.input_format), dest_fmt)
+
+            # Set the kernel
+        if args.set_kernel:
+            if args.set_kernel == '-':
+                language = notebook.metadata.get('jupytext', {}).get('main_language') \
+                           or notebook.metadata['kernelspec']['language']
+                if not language:
+                    raise ValueError('Cannot infer a kernel as notebook language is not defined')
+
+                kernelspec = kernelspec_from_language(language)
+                if not kernelspec:
+                    raise ValueError('Found no kernel for {}'.format(language))
+            else:
+                try:
+                    kernelspec = get_kernel_spec(args.set_kernel)
+                except KeyError:
+                    raise KeyError('Please choose a kernel name among {}'
+                                   .format([name for name in find_kernel_specs()]))
+                kernelspec = {'name': args.set_kernel,
+                              'language': kernelspec.language,
+                              'display_name': kernelspec.display_name}
+
+            args.update_metadata['kernelspec'] = kernelspec
+
         # Update the metadata
         if args.update_metadata:
-            log('[jupytext] Updating notebook metadata with {}'.format(args.update_metadata))
+            log("[jupytext] Updating notebook metadata with '{}'".format(json.dumps(args.update_metadata)))
             # Are we updating a text file that has a metadata filter? #212
             if fmt['extension'] != '.ipynb' and \
                     notebook.metadata.get('jupytext', {}).get('notebook_metadata_filter') == '-all':
                 notebook.metadata.get('jupytext', {}).pop('notebook_metadata_filter')
             recursive_update(notebook.metadata, args.update_metadata)
 
+            if 'kernelspec' in args.update_metadata and 'main_language' in notebook.metadata.get('jupytext', {}):
+                notebook.metadata['jupytext'].pop('main_language')
+
         # Read paired notebooks
         if args.sync:
             set_prefix_and_suffix(fmt, notebook, nb_file)
@@ -268,11 +314,11 @@ def writef_git_add(notebook_, nb_file_, fmt_):
             pipe_notebook(notebook, cmd, args.pipe_fmt, update=False)
 
         # III. ### Possible actions ###
-        modified = args.update_metadata or args.pipe
+        modified = args.update_metadata or args.pipe or args.set_kernel
         # a. Test round trip conversion
         if args.test or args.test_strict:
             try:
-                test_round_trip_conversion(notebook, args.to,
+                test_round_trip_conversion(notebook, dest_fmt,
                                            update=args.update,
                                            allow_expected_differences=not args.test_strict,
                                            stop_on_first_error=args.stop_on_first_error)
@@ -283,12 +329,12 @@ def writef_git_add(notebook_, nb_file_, fmt_):
 
         # b. Output to the desired file or format
         if nb_dest:
-            if nb_dest == nb_file and not args.to:
-                args.to = fmt
+            if nb_dest == nb_file and not dest_fmt:
+                dest_fmt = fmt
 
             # Test consistency between dest name and output format
-            if args.to and nb_dest != '-':
-                base_path(nb_dest, args.to)
+            if dest_fmt and nb_dest != '-':
+                base_path(nb_dest, dest_fmt)
 
             # Describe what jupytext is doing
             if os.path.isfile(nb_dest) and args.update:
@@ -305,9 +351,9 @@ def writef_git_add(notebook_, nb_file_, fmt_):
             log('[jupytext] Writing {nb_dest}{format}{action}'
                 .format(nb_dest=nb_dest,
                         format=' in format ' + short_form_one_format(
-                            args.to) if args.to and 'format_name' in args.to else '',
+                            dest_fmt) if dest_fmt and 'format_name' in dest_fmt else '',
                         action=action))
-            writef_git_add(notebook, nb_dest, args.to)
+            writef_git_add(notebook, nb_dest, dest_fmt)
 
         # c. Synchronize paired notebooks
         if args.sync:
@@ -359,21 +405,6 @@ def print_paired_paths(nb_file, fmt):
                 sys.stdout.write(path + '\n')
 
 
-def recursive_update(target, update):
-    """ Update recursively a (nested) dictionary with the content of another.
-    Inspired from https://stackoverflow.com/questions/3232943/update-value-of-a-nested-dictionary-of-varying-depth
-    """
-    for key in update:
-        value = update[key]
-        if value is None:
-            del target[key]
-        elif isinstance(value, dict):
-            target[key] = recursive_update(target.get(key, {}), value)
-        else:
-            target[key] = value
-    return target
-
-
 def set_format_options(fmt, format_options):
     """Apply the desired format options to the format description fmt"""
     if not format_options: