Allow joining and splitting on null bytes when calling system shell commands

[chtenb]

IPython has this very pretty way of calling shell commands:

files=!find . -name '*.cs' -o -name '*.cpp' -o -name '*.h' ! -path '*/obj/*'
print(files)

This is automatically split on newlines, such that files is of type SList. But if we change the above call to

files=!find . -name '*.cs' -o -name '*.cpp' -o -name '*.h' ! -path '*/obj/*' -print0
print(files)

which is actually more common, we end up with an SList with one string containing a lot of null bytes. I believe there is currently no way to have the output of the find command split on null bytes before it is stored in files. It would greatly improve the usability of IPython as shell script to have this as a feature IMHO, so there's my feature request. :)

It would be easy to add a .z attribute to SList etc, but I'm not sure about the combination with the ! syntax. Any thoughts?

Since there is no way to split on spaces using the ! syntax either, maybe it's just okay to settle for the .z properties. Would a PR implementing this be good for merge?

[takluyver]

The attributes on an SList are for rejoining it with different separators, rather than for splitting it. I don't think splitting on a null byte is common enough to build in, but it should be a one-liner where you want it:

files.n.split('\x00')

[chtenb]

The attributes on an SList are for rejoining it with different separators, rather than for splitting it.

Yes, and LSString has attributes for splitting.

I don't think splitting on a null byte is common enough to build in, but it should be a one-liner where you want it:

I'm a little surprised to hear this. Imho it is very common to do this. Many unix utilities, like find (-print0), grep (-z/-Z) and xargs (-0) support flags to print and consume output separated by null bytes, which people (should) always use when e.g. dealing with filenames (since filenames in unix systems can contain newlines) or with any kind of multiline records.

When IPython is used as a shell script replacing bash, utilities like this have to be as easy and convenient as possible.

As a side note, is it possible to obtain the output of a ! command as a LSString? It seems only SLists are returned by default.

[takluyver]

print and consume output separated by newlines, which people (should) always use when e.g. dealing with filenames

(I take it you meant null bytes instead of newlines) I understand the argument that this is the right thing to do, but I don't think I've ever seen someone actually doing it, or a shell script written like that.

As a side note, is it possible to obtain the output of a ! command as a LSString? It seems only SLists are returned by default.

I don't know of any built in way to do that. But you can easily convert it: LSString(x.n).

[chtenb]

but I don't think I've ever seen someone actually doing it, or a shell script written like that.

Okay, I see it all the time. As soon as filenames contain whitespace you need something like this. An alternative way to do it is by explicitly telling xargs to split on newlines, like `xargs -d '\n', but that I've never seen someone doing before :D

[Carreau]

This has been marked as no-action and had no comment in about a year. I'm going to close to try to keep the number of active issue reasonable. Feel free to reopen if needed.