shell command fails in IJulia

[raminammour]

Hello all,

I am using Jupyter 4.2.0, I execute this command in Julia 0.4 and 0.5:

;sustrip<src.su>src.rsf@
sustrip: subcalls.c: efopen: fopen failed

Whereas if I change the kernel to python:

!sustrip<src.su>src.rsf@

It works normally (as if executed from shell).

Your help is much appreciated!

[stevengj]

What do ;pwd and !pwd output in Julia and Python, respectively? Maybe they are not in the same directory?

Does it work if you do the same thing in the Julia REPL (i.e. the julia> command-line prompt, not IJulia)? If not, probably report a bug with Julia itself, not IJulia.

(What is the @ at the end of the filename for?)

[stevengj]

(Note that ls -F prints an @ at the end of symbolic links, but the @ is not actually part of the filename.)

[raminammour]

First, thanks for the reply!

1- same directory
2- Works in Julia REPL
3- The @ is part of the filename, but I had already tried with another filename with no special character (suspecting those may be a problem), it fails too.

Thanks

[stevengj]

Hmm, can you try:

Base.repl_cmd(`sustrip<src.su>src.rsf@`)

and

Base.repl_cmd(@cmd("sustrip<src.su>src.rsf@"))

to see if either works in IJulia? IJulia is transforming your line into the former, but I suspect we should be using the latter.

[raminammour]

Base.repl_cmd(`sustrip<src.su>src.rsf`,STDOUT)
sustrip: subcalls.c: efopen: fopen failed
Base.repl_cmd(@cmd("sustrip<src.su>src.rsf@"),STDOUT)
sustrip: subcalls.c: efopen: fopen failed

[raminammour]

Ok, some advance here, the command has a optional argument par=/dev/tty. Without it, it prints the parameters to screen (which is what python, and julia REPL do). Once I added a par file, the command worked! I still don't understand though why the REPL could handle it but not IJulia.

;sustrip <src.su outpar=par >src.rsf@

Feel free to close the issue if that is not something that impedes other users.

Thank you guys for your help!

[stevengj]

It probably has to do with the fact that STDOUT is a terminal device in the REPL, but is a pipe in IJulia (so that IJulia can capture output and redirect it to the browser). Maybe it is calling fopen on the pipe or something like that and failing?

Not sure if anything can be done here. cc @JobJob

[JobJob]

Sorry @stevengj, also missed this mention.

@raminammour does this work in IJulia?

Base.repl_cmd(`sustrip<src.su>src.rsf@`, IJulia.orig_STDOUT)

[raminammour]

@JobJob same error for above command

[JobJob]

Had a quick look at the code for Base.repl_cmd

It seems commands are run differently if STDIN is a TTY (which at the REPL it is, but in IJulia it isn't), see this line: $shell -i -c ... vs $shell -c ...

So I think that's the difference, not sure how ipython runs shell commands but doing this:

cmd = `echo hello`
shell = Base.shell_split(get(ENV,"JULIA_SHELL",get(ENV,"SHELL","/bin/sh")))
run(`$shell -i -c "($(Base.shell_escape(cmd))) && true"`)

gives me a warning:

bash: cannot set terminal process group (85462): Inappropriate ioctl for device
bash: no job control in this shell

So I don't think we want to mess with this.

I imagine if you replace my
cmd = `echo hello`
in the example above with
cmd = `sustrip<src.su>src.rsf@`
it should work but will maybe give you a warning. I'm curious now, so let me know if it does :)

I'm guessing this is more of an issue with shell commands that expect to be run interactively than with IJulia.

[raminammour]

It gives:

cmd =`sustrip<src.su>src.rsf@`
shell = Base.shell_split(get(ENV,"JULIA_SHELL",get(ENV,"SHELL","/bin/sh")))
run(`$shell -i -c "($(Base.shell_escape(cmd))) && true"`)

bash: no job control in this shell
sustrip: subcalls.c: efopen: fopen failed

I tried with different options for sh and had the same error too.

[raminammour]

I really think the problem is /dev/tty. I ran:

In Julia Repl:

;tty
/dev/pts/97

In IJulia from Jupyter:

;tty
not a tty

In Python from Jupyter

!tty
/dev/pts/80

Hope this helps :)

[JobJob]

Ok thanks. I think this is such a rare issue that I can live with it for now.

Thanks for reporting, it might help someone else one day.

[raminammour]

Thank you guys for your help!