Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

md to ipynb conversion does not create kernelspec in ipynb #230

Closed
psychemedia opened this issue May 17, 2019 · 12 comments

Comments

Projects
None yet
3 participants
@psychemedia
Copy link

commented May 17, 2019

i've been exploring using Jupyter Book for publishing from either .md or .ipynb files.

When using .ipynb files created from simple .md files on commandline using jupytext --to notebook *.md, Jupyter Book throws an error from not finding kernelspec data in the .ipynb (jupyter/jupyter-book#195).

If I load and save the notebook using Jupyter notebooks, a kernelspec element is saved into the .ipynb. Even if no specific kernel info is saved, should the Jupytext conversion from .md create empty kernelspec elements?

  "kernelspec": {
   "display_name": "",
   "language": "",
   "name": ""
  },
@mwouts

This comment has been minimized.

Copy link
Owner

commented May 17, 2019

Hello Tony, thanks for asking.

Can you give a try at the --update-metadata option? I have included an example below:

jupytext README.md --to ipynb --update-metadata '{"kernelspec": {"display_name": "", "language": "", "name": ""}}' -o - | tail -n 18
    "Contributions are welcome. Please let us know how you use `jupytext` and how we could improve it. You think the documentation could be improved? Go ahead! And stay tuned for more demos on [medium](https://medium.com/@marc.wouts) and [twitter](https://twitter.com/marcwouts)!"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "",
   "language": "",
   "name": ""
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
@psychemedia

This comment has been minimized.

Copy link
Author

commented May 17, 2019

That seems to work - thanks...

I think there is also a PR going defensive on the Jupyter-book side.

@mwouts

This comment has been minimized.

Copy link
Owner

commented May 17, 2019

Thanks Tony. By the way, please let me know if you need more adjustments for interoperability with Jupyter-book, that's a great project.

Is it correct that Jupytext + Jupyter-book will convert a (source) Markdown file to an Markdown file with outputs ? Do you think it would be interesting to mark the outputs in the Markdown file (say, with HTML comments), so that the source and output can be the same file ? I was considering trying something like this for #220...

@psychemedia

This comment has been minimized.

Copy link
Author

commented May 17, 2019

Jupyter Book converts ipynb or md to md thence html as part of a Jekyll publishing workflow.

I think it just renders what you give it rather than executing notebooks (I really should read the docs!).

I had a possible use case today around code outputs converting from a structured content (XML) format to md [about].

If the structured markup had been done right there would have been code and code output elements represented that would have been good to retain semantically in the md export, and thence render as such in a notebook generated from the md.

Generally, though, pretty much anything can appear as code output (also need to distinguish Out [] elements from eg print () output).

I need to play w/ Jupyter Book more esp. in context of Jupytext. eg if a Binder repo has ipynb,md Jupytext dualling enabled, it should emit md that could be copied straight into a github pages/jekyll _build dir?

Which is to say, there could be interesting publishing/editing worflows possible?

@psychemedia

This comment has been minimized.

Copy link
Author

commented May 18, 2019

@mwouts (Re: Jupyter Book, I'm just a user! (I'm never very sure about who's "allowed" to post to Github issues, so, erm, I increasingly do...))

@psychemedia

This comment has been minimized.

Copy link
Author

commented May 18, 2019

@mwouts "Is it correct that Jupytext + Jupyter-book will convert a (source) Markdown file to an Markdown file with outputs ?"

My workflow was:

  • route 1: convert XML -> md, then use Jupyter Book to just publish that md as if it were any md;
  • route 2: convert XML -> md, then use Jupytext to convert md to ipynb; then let Jupyter Book generate md from the ipynb and then publish the md to a static site. (Actually, I think Jupyter Book may (be able to) execute the ipynb files and use md export containing cell output from that... Hmmm... Need to check that. This suggests it does / can. )

What might be interesting would be a Jupyter Book switch to allow Jupyter Book to use Jupytext to turn md into ipynb, execute the ipynb, generate some md with code cell outputs, then publish the static site?

@mwouts

This comment has been minimized.

Copy link
Owner

commented May 19, 2019

I'm never very sure about who's "allowed" to post to Github issues, so, erm, I increasingly do...

In this project at least, issues are welcome! With them I've learnt so much, people told me about there use case, overall the interaction has been extremely positive... By the way I still do not have contributing guidelines here, so maybe I should make it clear that issues and PR are welcome... I will do that soon.

What might be interesting would be a Jupyter Book switch to allow Jupyter Book to use Jupytext to turn md into ipynb, execute the ipynb, generate some md with code cell outputs, then publish the static site?

I agree, there's a series of cases in which we want to convert a text file to a notebook, and execute it. I plan to see what we can do at #231 - stay tuned !

@mwouts mwouts reopened this May 22, 2019

@mwouts

This comment has been minimized.

Copy link
Owner

commented May 22, 2019

I am hitting the same issue for #231... Papermill will not take a notebook with no kernel information.

It should be reasonable to set the kernel that matches the current python env if the notebook is a Python notebook, and otherwise set the first kernel for the language being used. Let's see if we can do that...

@nathancarter

This comment has been minimized.

Copy link

commented May 24, 2019

Just recall that .py is one of several formats that jupytext supports. For instance, if the input is .md instead, then you'll need to set the kernel on a code-chunk-by-code-chunk basis.

@mwouts mwouts added this to the 1.1.4 milestone May 25, 2019

mwouts added a commit that referenced this issue May 27, 2019

@mwouts

This comment has been minimized.

Copy link
Owner

commented May 27, 2019

@psychemedia , Version 1.1.4 will have a new --set-kernel argument to set the kernel for files that do not have a Jupytext header. A preview of that version is available with

pip install jupytext==1.1.4-rc1

@nathancarter , thanks for your comment. Jupytext creates a Jupyter notebook from the .md file, so unfortunately the document can use only one kernel. Jupytext will choose the kernel that matches the language of most of cells. Still, the language for the other cells is not lost: we use the magic commands like %bash or %R in that case.

@nathancarter

This comment has been minimized.

Copy link

commented May 27, 2019

Oh, right, I had forgotten that a notebook is associated with one kernel. Thanks.

mwouts added a commit that referenced this issue May 30, 2019

mwouts added a commit that referenced this issue Jun 4, 2019

mwouts added a commit that referenced this issue Jun 4, 2019

@mwouts mwouts referenced this issue Jun 4, 2019

Merged

1.1.4 #243

mwouts added a commit that referenced this issue Jun 5, 2019

mwouts added a commit that referenced this issue Jun 5, 2019

@mwouts

This comment has been minimized.

Copy link
Owner

commented Jun 5, 2019

Version 1.1.4 is available on pypi, and has the --set-kernel argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.