From rmagic to an R console

[lgautier]

This is a wish, suggested avenue for a new feature, and may be a possible GSoC project, rather than a bug report.

There are situations where it would be nice to be able to switch from a extended console for Python (ipython) to an R console with autocompletion. It could be useful for:

• R users trying out Python (they can fall back on the R console whenever they like)
• any Python user knowing R (nothing like )
• it could also be used by R-only users if for the ipython notebook (the RStudio + knitr combo might already be taking care of that crowd; if the case no point in focusing on those users).

[Carreau]

Please have a look at #2854.
It should let you create a R kernel for IPython if you wish.

and may be a possible GSoC project
It has been discussed on the ML and probably not this year.

And why stick to R. We should be able to get any kernel hooked. The only thing is you will not be able to have multi kernel in the same notebook, but you could have a notebook that is an r kernel or a ruby kernel .... etc.

[lgautier]

Thanks.

AFAIUI, although the ability to create kernel is part of the answer, and a general answer to providing a console or notebook to other languages, but that does not seem to be all.

Emulating the R console (or even better, extend it) would be needed.

I have intent of restricting all to R, but that proposal is about providing an R console. A "universal console" sort of project, with a declarative layer to quickly fit any language, would be an interesting idea but going well beyond the scope of a GSoC-sized project, may be.

[Carreau]

Sorry, I'm not very familiar with R ( I know I should), what do you mean by :

Emulating the R console (or even better, extend it) would be needed.

If you are referring to completion, rich repr, introspection ... as long as you conform to ipython protocol you get that for free.

[lgautier]

The R console is a REPL environment, somewhere between the default Python console and bpython.

If get it so far, the bottom line is that there is infrastructure (kernels) to do what my initial message above described. This is making the proposal something very reasonable for a GSoC.

[Carreau]

The R console is a REPL environment, somewhere between the default Python console and bpython.

If get it so far, the bottom line is that there is infrastructure (kernels) to do what my initial message above described. This is making the proposal something very reasonable for a GSoC.

Having a R kernel for IPython would be great, But I don't think none of us have enough time to mentor a student to do so. Still if you find someone and need help feel free to ping us.

[lgautier]

I certainly do not have much time, but might be comfortable on the Python & R side of things. If this is about implementing a documented interface, and if the right student can be found it might just fly.

Is ipython applying to GSoC ? (I could certainly apply for rpy2 as an organisation, but that'd seem more effort than just a project in ipython).

[Carreau]

Is ipython applying to GSoC ? (I could certainly apply for rpy2 as an organisation, but that'd seem more effort than just a project in ipython).

No, you can look on the ML and most of the involved people can't. Either because there are requirement with the Sloan grant, or because we are ourselves student and/or don't have time to be a mentor.

We'll soon release a roadmap for the next 2 years and what would be the things that can be tackled by the core team and what will not. Allow easier custom kernel of other language is on the list, but writing them is definitely not.

Also things like separate kernel will gain a lot (especially at the beginning) to be separate. IPython is big and slow to release, and PR will take time to merge. Still if you write a different kernel, we'll be happy to advertise it.

[lgautier]

Bummer that the grant came with those strings attached.

It makes sense that the kernels (and their maintenance) are considered third-party additions to ipython. I did not suggest that the outcome would end up in the main codebase.

One way to look at it would be to say "wait for the custom kernel" part to appear before proceeding, but on the other hand it will very likely be that trying to implement a custom kernel for an other language will provide input for making the implementation of other languages easier.

I'll see if SciPy would host that, otherwise that will be for an other year (or an unpaid motivated individual stepping forward).

[Carreau]

One way to look at it would be to say "wait for the custom kernel" part to appear before proceeding, but on the other hand it will very likely be that trying to implement a custom kernel for an other language will provide input for making the implementation of other languages easier.

Yes totally agree, and we'd love having someone doing that. We already have a ruby one that helped us a lot in changing stuff.

I'll see if SciPy would host that, otherwise that will be for an other year (or an unpaid motivated individual stepping forward).

That would be great, and I think it would make even more sens to have something like that that came from the R community. And we totally target the ability to have notebook that would be R by default.

[lgautier]

Then the best bet is GSoC (under a project TBD) + right student.

I am not completely convinced about a push for this coming from the R community, since 1) RStudio is already looking at notebook type of functionalities, 2) the R community more able to provide R skills than Python skills (the later being needed here).

Here the motivation to implement this kernel while RStudio has started an effort on notebooks would be to allow:

• the implementation of custom notebook-like services (built on the top of iPython)
• all sorts of polyglot and eventually distributed applications (AFAIUI, kernels can be remotely connected to)... some kind of "Thrift for REPL environments")

[Carreau]

Short from my phone.

There is a missconception here. A R/ruby/whatever kernel does not need
python at all. It could even run on a different machine than the server.
And the end user could totally be unaware that python is involved in the
end.

We could perfectly at some point replace the python server with lisp and
nobody would be able to see the difference.
Le 17 févr. 2013 18:52, "Laurent Gautier" notifications@github.com a
écrit :

Then the best bet is GSoC (under a project TBD) + right student.

I am not completely convinced about a push for this coming from the R
community, since 1) RStudio is already looking at notebook type of
functionalities, 2) the R community more able to provide R skills than
Python skills (the later being needed here).

Here the motivation to implement this kernel while RStudio has started an
effort on notebooks would be to allow:

• the implementation of custom notebook-like services (built on the
  top of iPython)

• all sorts of polyglot and eventually distributed applications
  (AFAIUI, kernels can be remotely connected to)... some kind of "Thrift for
  REPL environments")

  —
  Reply to this email directly or view it on GitHubhttps://github.com/From rmagic to an R console #2947#issuecomment-13689914.

[lgautier]

Thanks. I had understood correctly the part about kernels being possibly remote (as in "run on separate machines), but I confess that I have been lacking a digest about what a "kernel" really is (pointers to documentation are welcome).

Now, from that last comment is a "kernel" seems to be an application listening and emitting messages (with an ipython-specific communication protocol). That would mean that an R kernel as you think of would be R code implementing that communication protocol (and run an application that uses it).

The proposal here is that, whatever a kernel is, there is an implementation of it for the default interface to Python in iPython. Now, since rpy2 can embed R into a Python process and basically make the communication between both seamless, the idea was to use that for making an R console usable from iPython with a close to minimal effort. From the top of my head what would need to be tweaked is:

• syntax highlighting (I contributed to pygments a lexer for R/S years back and it is still there, that should be enough)
• autocompletion (expected to be minimal)
• help system (rpy2 has feature to fetch R's man pages)
• magics (In R, '%' can used for operators)

[Carreau]

messaging protocol.

Seeing how magics works, (with strings) hooking will be complicated IMHO.

[takluyver]

On 23 February 2013 09:30, Laurent Gautier notifications@github.com wrote:

magics (In R, '%' can used for operators)

Note that there's no requirement that a kernel implements magic commands.
The IPython kernel supports the Python language plus IPython's special
syntaxes like magics, but you could just as well have a pure-Python kernel

• in fact, we used to, but it wasn't well tested or maintained. So an R
  kernel could just provide R syntax.

(Personally, I find the integration of other languages within the IPython
kernel, using cell magics, more interesting than separate standalone
kernels. But I can imagine that they'll be valuable to others.)

[Carreau]

(Personally, I find the integration of other languages within the IPython
kernel, using cell magics, more interesting than separate standalone
kernels. But I can imagine that they'll be valuable to others.)

I don't think they have the same goal.

Especially with the cell magic you can push/pull variable.
Having other kernel, will/should help having a more language agnostic machinery and can help gather developer.

[lgautier]

Wouldn't a "digest for the uninitiated" of the concepts and goals behind "cells", "magics" and "kernel" be a good addition to the documentation ?