Bash to xonsh translator

[anki-code]

Originally posted by scopatz in #1614 (comment) and #1614 (comment):

If we wanted to be clever we'd build a sh-parser and translator to xonsh...
Well, I think having something like this would still be valuable, even if it wasn't a fully featured translator. If anyone wants to work on it, I would be most grateful!
Not now, of course, there are still too many open issues :)

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

[anki-code]

There is bashlex - Python parser for bash. The parser allows to create AST for bash and used in http://www.explainshell.com/.

Example:

pip install bashlex
import bashlex
parts = bashlex.parse('echo 123\nfor VAR in 1 2 3\ndo\n echo $VAR\ndone')
for ast in parts:
      print(ast.dump())

#CommandNode(pos=(0, 8), parts=[
#  WordNode(pos=(0, 4), word='echo'),
#  WordNode(pos=(5, 8), word='123'),
#])
#CompoundNode(list=[
#  ForNode(pos=(9, 44), parts=[
#    ReservedwordNode(pos=(9, 12), word='for'),
#    WordNode(pos=(13, 16), word='VAR'),
#    ReservedwordNode(pos=(17, 19), word='in'),
#    WordNode(pos=(20, 21), word='1'),
#    WordNode(pos=(22, 23), word='2'),
#    WordNode(pos=(24, 25), word='3'),
#    ReservedwordNode(pos=(26, 28), word='do'),
#    CommandNode(pos=(30, 39), parts=[
#      WordNode(pos=(30, 34), word='echo'),
#      WordNode(pos=(35, 39), word='$VAR', parts=[
#        ParameterNode(pos=(35, 39), value='VAR'),
#      ]),
#    ]),
#    ReservedwordNode(pos=(40, 44), word='done'),
#  ]),
#], pos=(9, 44))

# Prepared by xontrib-hist-format      

This could be the start point to creating simple translator from bash to xonsh along with Bash to Xonsh Translation Guide and Bashisms xontrib.

Note! Apple replaces bash with zsh as the default shell in macOS Catalina in 2019 and now Kali Linux repaces bash with zsh in 2020. It will be great to create this not only for bash but zsh also.

[gforsyth]

Thanks for raising this @anki-code -- are there any language differences between bash and zsh? Or is it just that zsh comes with additional builtins?

[anki-code]

The answer on "Are all bash scripts compatible with zsh?":

Zsh has most features of bash, but in many cases with a different syntax.

I think that a good approach is to test bash commands that will be supported by translator to xonsh in zsh also and describe the difference if it's exists.

[Shellcat-Zero]

There is bash2py, which might provide a decent starting point for this.

Does anything exist for Xonsh to Bash, or is that similarly feasible? From a sysadmin perspective, I can think of a lot of cases where dumping Xonsh/Python to Bash scripts would be very useful for some target environments.

[anki-code]

bash2py

Nice tool, thanks!

Does anything exist for Xonsh to Bash, or is that similarly feasible?

Nope. I think it's feasible :)

[Techcable]

Just to be clear, this is something I'm offering to work on (as long as you're happy with my general approach).

I'm just warning in advance that my solution will probably spawn a lot of subprocesses ;)

EDIT: I have abandoned the old zsh2xonsh solution. I retained the above comment for historical interest. I have some thoughts on a more robust/complete solution....

[anki-code]

Xonsh has functionality for sourcing from foreign shells (source-bash, source-zsh). But this functionality works only for simple cases and users create issues around wrong sourcing.

I believe that we should close all of these issues with the reference to this message and this issue to focus all of these issues on creating more valuable translators from bash/zsh to xonsh that will cover the simple use cases as well as complex use cases.

[jaraco]

One concern I have with the proposed approach is portability. One of my main reasons for adopting xonsh over powershell or zsh is it's decent portability (first-class support for Windows/Mac/Linux). I'd like for any solution to be portable or at least have a clearly documented boundary where dependencies (zsh, bash) are required.

[Techcable]

One of my main reasons for adopting xonsh over powershell or zsh is it's decent portability (first-class support for Windows/Mac/Linux)

100% this. The ideal solution has first class support for Windows and Mac.

or at least have a clearly documented boundary where dependencies (zsh, bash) are required.

There's a good chance that our solution will be less than ideal (shells are not known for their elegance). In this case we will need to clearly document any dependency/platform requirements and have graceful fallback.

A wrapper around an existing shell (zsh/bash) would be Inherently POSIX-specific.

EDIT: I am planning a post on the options I see for translation. I have essentially abandoned my zsh2xonsh approach

[anki-code]

shfmt - A shell parser, formatter, and interpreter with bash support (zsh will be supported soon).

I've tried shfmt and it can parse bash and return syntax tree in json format :

cd /tmp
open https://go.dev/doc/install  # Install go
$PATH.append('/usr/local/go/bin')  # macos
$PATH.append('~/go/bin')  # macos
go install mvdan.cc/sh/v3/cmd/shfmt@latest  # https://github.com/mvdan/sh
wget https://raw.githubusercontent.com/mvdan/sh/master/syntax/canonical.sh  # example

cat canonical.sh | shfmt --to-json > canonical-tree.json  # Get syntax tree
cat canonical-tree.json | jq -c

So it's possible to get this syntax tree and translate into xonsh syntax.

[anki-code]

I'm going to close this but feel free to continue comment this intention.

[bestlem]

Add to similar things - there is babelfish A go executable - I think it would be possible to use the code to generate xonsh rather than fish