%%writefile magic or similar for IRkernel?

[breschke]

The %%writefile magic in Jupyter is a major component of my workflow: I trial code in the notebook, then write out the final version to .py for future use. (I don't anticipate opening a notebook every time I want to run a particular module.) I'm missing the convenience of this magic in the IRkernel. Any pointers/plans for future development?

[takluyver]

Sorry, magics are a feature of IPython, the Python kernel, not Jupyter itself, and we have no plans to implement something similar for IRkernel.

However, if you want to write code in a notebook and then convert it to an R script, you can use nbconvert to convert the whole notebook. There should be an option in the notebook menu (File, Download as), or you can run at a command line jupyter nbconvert --to script my_notebook.ipynb.

[breschke]

Hmm. I think this will inhibit the use of IRkernel for rapid code prototyping.

My best workaround at the moment is to use the writeLines function (making sure to use double quotes strictly in coding). e.g., preface the target cell with one containing the following:

con <- file("/path/file.R")
writeLines('

[target cell]
followed by:

',con)

then merging the three together and executing. This is little faster than copying and pasting into an editor.

I'm familiar with the Download as and nbconvert routes, but I often don't want all of the cells in a resulting script. The intent is to document how I authored a given script--i.e., striving for Jupyter's core tenet of literate programming.

[abalter]

What would it take to implement this by capturing the %% character when at the start of a line and then doing something with it?

More specifically, if the python kernel has magics, what would it take to add them ti the R kernel?

[flying-sheep]

It’s less about it being hard and more about it being unnecessary, this is easy enough, no?

cat('
You only have to escape typewriter quotes \'
Multiple lines are fine
', file = 'thing.txt')

[abalter]

@flying-sheep -- Sorry I want to have an R notebook and do

%%python
[x for x in range(5)]

or

%%bash
!echo hello

[flying-sheep]

I think using reticulate or system is pretty easy, no?

[abalter]

1. Can you show how you would do this in a notebook?
2. Maybe there are 100 workarounds. Why not just have the same functionality in all jupyter kernels?

[takluyver]

Why not just have the same functionality in all jupyter kernels?

The magics system is part of IPython, not Jupyter. It's up to the people working on each kernel whether they want to support some kind of extra convenience syntax as well as the programming language the kernel is for. Please don't approach this with an expectation that every kernel should reimplement magics.

That said, if people do want to, cell magics (%% on the first line) are by far the easier ones to implement. Line magics (% on any line) are really messy.

[abalter]

@takluyver -- exactly. And,I think that is why this issue is in IRkernel, not Jupyter. At least, I would think that IRkernel would be the R equivalent of IPython. If magics were going to be implemented for R, it would be here, right?

[takluyver]

Yep, this is where it would go. But I want to set expectations correctly: a kernel is an execution backend for a programming language, with no expectation that it extends that language. Please don't show up to kernel maintainers (largely @flying-sheep now for IRkernel; I haven't touched it for years) and expect them to 'just' provide IPython features. Approach this as a new feature you are hoping to convince someone else is worth implementing, maintaining, and dealing with bug reports on, not something that the project is self-evidently missing.

Of course, like with any open source project, you can offer to help implement & maintain it, which means it's less work for the existing maintainers. But reviewing PRs takes time & energy too, and it's hard to be sure that a new contributor will stick around for maintenance, so you're still asking something of the maintainers.

[abalter]

Gotcha. The first thing would be to look at the code for the %% magics in the IPython kernel and see how it is implemented there.

[abalter]

After a quick look, the magics are in IPython.core.magic. So, as you stated, they are part of IPython, not ipykernel.

This suggests to me that down the road a more modular architecture would be that IPython would be generalized to a notebook library (perhaps Jupyter itself) with hooks for any kernel. Then the syntax of magics would be the same for all kernels, and magic could be used for any kernel.

The notebook driver (for now IPython) would be kernel agnostic.

Is something like this on anyone's radar, or is it worth a feature request to Jupyter or IPython?

[takluyver]

There have been some efforts to generalise kernel features - see metakernel. But AFAIK you have to build the kernel in Python to use that, whereas Jupyter as a whole is designed so that kernels can be implemented in any language. You could propose some extra layer between the frontend and the kernel which would intercept & handle certain code cells, but doing that as an official Jupyter feature would be a pretty big step.

[krassowski]

I would love to have magics in IRkernel! It seems that at least the xeus-based R kernel alternative intends to implement those, though they are not active recently. I extensively reviewed the magics adoption across Jupyter kernels here. About workarounds - probably none will be as good as having dedicated magics, but maybe you could consider using the IJulia solution of showing the workaround when user tries to use magic?