New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ob-async should support :session header-arg #1

Open
astahlman opened this Issue Feb 16, 2017 · 10 comments

Comments

Projects
None yet
7 participants
@astahlman
Owner

astahlman commented Feb 16, 2017

Desired behavior

#+BEGIN_SRC sh :session foo :async
export GREETING="Hola"
#+END_SRC

#+RESULTS:


#+BEGIN_SRC sh :session foo :async :results output raw
[[ -n "$GREETING" ]] && echo "$GREETING" || echo ":session is not respected"
#+END_SRC

#+RESULTS:
Hola

Current behavior

#+BEGIN_SRC sh :session foo :async
export GREETING="Hola"
#+END_SRC

#+RESULTS:


#+BEGIN_SRC sh :session foo :async :results output raw
[[ -n "$GREETING" ]] && echo "$GREETING" || echo ":session is not respected"
#+END_SRC

#+RESULTS:

:session is not respected
@stsquad

This comment has been minimized.

Contributor

stsquad commented Feb 23, 2017

Can async support this? I thought :session was meant to run things in a inferior process which should be asynchronous anyway?

@drym3r

This comment has been minimized.

drym3r commented Mar 28, 2017

Can async support this? I thought :session was meant to run things in a inferior process which should be asynchronous anyway?

No, it doesn't. emacs-async opens an independent emacs process which executes all that's in the block. What could be done, maybe, is writing all the evaluated stuff and every time something is evaluated, that file is readed and evaluated on the independent emacs process. Something like this workflow:

I execute this:

#+BEGIN_SRC sh :session foo :async
export GREETING="Hola"
#+END_SRC

The independent emacs process evaluates it and writes the result on /tmp/something/foo, for example. Then, when we execute this:

#+BEGIN_SRC sh :session foo :async :results output raw
[[ -n "$GREETING" ]] && echo "$GREETING" || echo ":session is not respected"
#+END_SRC

Before executing it, /tmp/something/foo could be sourced. I guess it's not that easy and maybe it won't be usefull in all cases or languages, but maybe in some yes.

@astahlman

This comment has been minimized.

Owner

astahlman commented Mar 28, 2017

Hmmm, yeah - that's a good point. If we were to serialize the session to a file and then source it in an independent process then the contents of the blocks would need to be idempotent.

I wonder if it would be possible to start a persistent session in the child process with async-start and send it the contents of src blocks for evaluation with async-send.

@drym3r

This comment has been minimized.

drym3r commented Mar 29, 2017

That may be a good way to do it. That's how python or bash :session works (I believe), it creates a background process and feeds them with all the commands.

@astahlman

This comment has been minimized.

Owner

astahlman commented Feb 14, 2018

I wonder if a thread-based implementation (supported in Emacs 26) would make this easier? See this comment for details.

@preetpalS

This comment has been minimized.

preetpalS commented Feb 15, 2018

This following comment was based on the belief that Emacs 26 would allow multiple threads to execute at once (this is not the case). See discussion following jwiegley/emacs-async#97 (comment).

@astahlman I'm not completely familiar with the semantics of sessions (I am assuming they are only for interpreted languages (except Emacs Lisp? See ob-emacs-lisp.el in https://code.orgmode.org/bzg/org-mode/src/master/lisp)) but I believe that you could view sessions as being uniquely accessed using a composite key consisting of the session name and programming language.

There are two paths (that I can think of right now) that you could take to implement sessions:

  1. Integrate with existing sessions support in upstream org babel supported languages.
  2. Independently support sessions (this would be confusing since sessions of the same name would not be shared between async and synchronous code blocks).

The following discussion, assumes option 1 is the only viable option.

Assuming that sessions are lazily initialized (see https://code.orgmode.org/bzg/org-mode/src/master/lisp/ob-ruby.el#L109 ; for ob-ruby they are), you would probably need mutexes to guard against concurrent initialization of sessions. You would also probably need to synchronize access to the sessions also. There might be issues if sessions are currently being created in a way that any interactions with them block the main thread. I don't think that this library can integrate sessions without coordinating with upstream.

@astahlman

This comment has been minimized.

Owner

astahlman commented Feb 21, 2018

@preetpalS Thanks for looking into it, any rate. The discussion in jwiegley/emacs-async#97 was enlightening.

@fkgruber

This comment has been minimized.

fkgruber commented Jun 30, 2018

For R code blocks the :session support is just critical. I can't see much use otherwise.
Anyway fantastic for shell blocks and others.

@nico202

This comment has been minimized.

nico202 commented Jul 10, 2018

Any luck @jkitchin is interested in helping in this? :pray_tone:

@jkitchin

This comment has been minimized.

jkitchin commented Jul 10, 2018

This might already work with ob-ipython (which can support other kernels via Jupyter). In the scimax modifications to ob-ipython (https://github.com/jkitchin/scimax/blob/master/scimax-org-babel-ipython-upstream.el) there is support for async execution that works for ipython, but I guess should work for R too, with an R jupyter kernel. I haven't used other kernels very extensively though, so I don't know if there will be issues with it.

@nico202 nico202 referenced this issue Jul 17, 2018

Closed

Problem interacting with ob-julia #37

1 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment