syntax: ksh support

[jansorg]

This is a proposal to add support for ksh, alongside mksh.

Why I think that ksh support would be good:

• imo a common setup is shfmt + ShellCheck. ShellCheck supports ksh. To allow to format the files, which were previously validated, would be quite useful.
• formatting ksh files currently breaks the source, because function name {...} is turned into function name() {...}, which ksh is unable to parse.
• (I'm biased, of course, because I'm bundling shfmt with BashSupport Pro and allowing to format .ksh files would be useful)

Differences:

• ksh only support name() {...} and function name {...}, and does not support function name() {...} declarations.
• some non-technical differences are listed at https://www.mirbsd.org/mksh-faq.htm#kornshell
• I couldn't find a reference of technical differences. For ksh, it's most likely a subset of things supported by mksh.

I'm willing to contribute to ksh support.

Unclear to me:

• I assume that interp isn't supported to support all the dialects, as it seems to implement POSIX.

[mvdan]

Sounds good to me. It sounds like it shouldn't be too much work, beyond carefully altering the logic for what should be allowed in ksh vs mksh.

The interpreter, for now, doesn't have a "language mode" like the parser does. For the most part, a syntax.Node should be interpreted the same way regardless of the shell. Many node types or fields only appear in some dialects anyway, like ArrayExpr can never be parsed in POSIX mode as it doesn't have arrays.

It is true that shells behave in different ways, and even have POSIX-compatible and POSIX-incompatible modes like Bash does, but I honestly haven't had the need to implement that just yet.

If someone needs that level of detail, we can discuss it in an issue and possibly implement it.

[mvdan]

So, to further work on this, one would need to:

• Add an extra language variant
• Add tests which execute ksh, modifying or adding test cases specific to ksh
• Modify the parser or printer as necessary

[mvdan]

I'm actually about to push a very similar change for bats support in #600, so you could copy a lot of the mechanism.

[mvdan]

I've delayed v3.3 enough, so I'm going to release it without ksh support for now. This is still on my radar, though.

@jansorg After 943b9d0, are there any ksh programs that shfmt fails to parse or prints incorrectly?

[jansorg]

@mvdan I did not receive any other report about incompatibilities.

[mvdan]

Great, so it sounds like we could add initial ksh support as just a variant of mksh with minimal to no other changes.

What ksh version is maintained these days, and used by the most people? Arch has 2020.0.0, so that's a start.

[ko1nksm]

Hi,

In my opinion, the most widely used version today is ksh 93u+. ksh 2020 was released once, but is believed to have been abandoned because it failed to solve the problem. As far as I know, the currently maintained version is ksh 93u+m (not yet released). see below.

• https://github.com/ksh93/ksh (ksh 93u+m)
• https://en.wikipedia.org/wiki/KornShell

[mvdan]

Thank you @ko1nksm, that at least confirms my suspicion that 2020 is not a linear successor to 93.

Sounds like, for now, we should use the last official release: ksh93u+ in 2012.