Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support zsh #809

Open
eigengrau opened this issue Dec 25, 2016 · 31 comments

Comments

Projects
None yet
@eigengrau
Copy link

commented Dec 25, 2016

So, I’ve noticed quite a while ago that zsh support had been removed. So, since christmas is a time for miracles, I thought I might check back on any recent developments. From searching the issue tracker I take it that nothing has changed in this regard. So I figured I might open this issue so that patient, interested parties can subscribe to it and get notified in case someone feels like stepping up.

That is of course if you do deem zsh support generally useful. Otherwise please feel free to close the issue.

Thanks for making shellcheck!

@HaleTom

This comment has been minimized.

Copy link

commented Jan 16, 2017

shellcheck is amazing, and it is the reason I still write most of my scripts in bash.

I'd love it if zsh support could be added though!

@molovo

This comment has been minimized.

Copy link

commented Feb 1, 2017

As someone with quite a few big ZSH projects out in the wild, I'd definitely be interested in seeing this happen. I'd be very happy to contribute myself if someone could let me know what would be required to get it implemented.

@bittorf

This comment has been minimized.

Copy link

commented Feb 3, 2017

@molovo: first you have to ask yourself if the features of 'zsh' are a subset of other shells or if zsh has own features. look into the source for 'first citizen shells' and the handling of 'dash' and 'bash' and you will get a feeling, what is needed...

@koalaman

This comment has been minimized.

Copy link
Owner

commented Feb 3, 2017

This would require:

  1. Implementing all the syntax that is unique to zsh, to allow scripts to be successfully parsed (Parser.hs). This requires some care to avoid strange parse errors for other shells.
  2. Extend the variable flow tracing code and similar to support these new structures, so that shellcheck can tell when variables are modified and such (AnalyzerLib.hs).
  3. Fixing or extending any tests that are not correct for zsh (Analytics.hs, Checks/*.hs).
  4. Adding new and interesting Zsh specific checks.

Ksh shares basically all its syntax with Bash, which is why it's still supported even though almost no one uses it. Zsh required a lot more work for a similarly small number of users, which is why I unfortunately wasn't able to keep it up.

@koalaman koalaman referenced this issue Mar 25, 2017

Closed

Cannot parse EXTENDED_GLOB #855

1 of 2 tasks complete
@LukeShu

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2017

If zsh is ever supported, it should recognize #compdef and #autoload as shebangs for zsh, as these are magic headers used in zsh autocompletion helpers.

@kgizdov

This comment has been minimized.

Copy link

commented Mar 30, 2017

@koalaman I understand that bash is way more popular that zsh, but it still has a huge community. Would it be appropriate if you make a project/branch for zsh contributions so that users can contribute. Maybe put a note in the README and helpwanted flag here. If you set up the infrastructure, I'm sure people will bunch up to make this happen. I am willing to spare some time on this.

@brother

This comment has been minimized.

Copy link

commented Mar 31, 2017

@kgizdov on the other hand a PR with the start is pretty much the same. I assume that if you gather some people to do the work either they are welcome to the project or they will fork it.
The code won't be written and maintain by pure magic, as far as I can understand @koalaman will not take that responsibility himself.

@paulmelnikow

This comment has been minimized.

Copy link

commented Mar 31, 2017

I looked into this briefly a couple weeks ago. After I read through most of Parser.hs it was apparent that koalaman's step 1 would require deep knowledge of zsh syntax and bash syntax and a decent understanding of Haskell.

That made me wonder if zsh's parser (it's in C) could be used directly or modified slightly to emit an AST that could be interfaced to some high-level language constructs in Haskell or some other language. The wrapping is a bit involved, but is less work than reimplementing the parser.

I'm curious, has anyone on this thread worked with the zsh source?

@molovo

This comment has been minimized.

Copy link

commented Mar 31, 2017

@paulmelnikow I haven't worked with the source, but I have considered building an AST or something similiar (not for this purpose, but to support a coverage driver for zunit). I wonder if something could be built to tackle both issues at once?

@paulmelnikow

This comment has been minimized.

Copy link

commented Mar 31, 2017

@molovo Interesting idea!

@snoblenet

This comment has been minimized.

Copy link

commented Sep 21, 2017

Maybe the wiki could recommend a shellcheck alternative for zsh users? Because all Google searches based on variants of lint zsh take you back to shellcheck.

@paulmelnikow

This comment has been minimized.

Copy link

commented Sep 22, 2017

Unfortunately, there is no such equivalent. If anyone could make it happen, that'd be @molovo.

You may want to take a look at fish shell, which is better documented, more user-friendly, and developed on github.

@ghost

This comment has been minimized.

Copy link

commented Oct 13, 2017

I would like a support for Zsh but well Bash is still pre-installed in all major distro.

I code in Bash using Zsh...haha

@davidosomething davidosomething referenced this issue Nov 15, 2017

Merged

Undo 712c3ac zsh>sh + shellcheck #1733

1 of 1 task complete

Ferenc- added a commit to Ferenc-/dotfiles that referenced this issue Feb 20, 2018

zsh: Add bash shebang until shellcheck doesn't support zsh
The github issue is the following:
koalaman/shellcheck#809
Maybe an alternative linter could be used...
@darthdeus

This comment has been minimized.

Copy link

commented Jul 24, 2018

I guess one could still use shellcheck for bash-ish zsh files? To be more specific, I have quite a few config files related to my ~/.zshrc and that's about the only place where I use zsh.

But being completely honest, I'm way more familiar with bash than with zsh, so sometimes when shellcheck is showing a warning, I'm not 100% sure if it applies to zsh as well.

@psprint

This comment has been minimized.

Copy link

commented Aug 11, 2018

I see Zsh thing happening if:

  1. Shellcheck will be forked to e.g. ZShellcheck.
  2. The authors do some effort to clean up the source to allow a fresh start for Zshell extensions, e.g. authors would disable support for ${!VAR} bash indirect-reference syntax and other things like this. It might be that some Bash property/feature required a hack or a bloated code somewhere, and this could be removed now.
  3. A good code documentation will be available, and a channel to receive help from the authors.
  4. Some people will get around the forked project, start small by e.g. implementing support for autoload functions (a feature of Zsh).
  5. This way original shellcheck will sleep well, being undisturbed with Zsh-universe related actions.

I write this because I'm 3-years active Zsh (upstream) and Zsh-script-projects developer, and the above scenario covers my actual expectations to have a well organized environment/setup for working on a project.

BTW., Zsh can parse its code itself with a (z) substitution flag, I have based on it a Doxygen-like tool that works also with Bash scripts, check out significant-large Bash project (not mine) that uses it for Bash backend-scripts (the link points to a code-documentation generated with Zshelldoc). A code-documentation tool is somewhere in direction of linter tool.

@HaleTom

This comment has been minimized.

Copy link

commented Aug 27, 2018

There is so much commonality between {ba,z}sh that there would need to be a lot of importing upstream commits into the new project... @psprint What do you see as the advantages of a forked project?

Would it make sense to:

  1. Move all common {ba,z}sh syntax to a meta-syntax, for example named: bazsh
  2. Implement only the bash-specific extensions (eg things like ${var@Q}) in bash
  3. Start the zsh-only syntax with a clean slate (importing only the common bazsh syntax)?

(I'm assuming one syntax can import/inherit from another syntax here - please correct me if I've misunderstood)

@psprint

This comment has been minimized.

Copy link

commented Aug 27, 2018

I think that without fork programmers will have to be real experts in Bash and Zsh. I learn Zsh for 3 years and sometimes there's still something that's new to me. If I imagine that I now would have to reach the same level in Bash but not through play & experiment like with Zsh, but with studying documentation, then I feel like running away from contributing.

My experience also tells me that structural programming doesn't match situation, where every action that a program has to make is also a result of condition like "Bash or Zsh". If I imagine codebase that's doing constant "Bash or Zsh" checking, I see something really severe, a mess. This mess can be controlled but this doesn't match Open Source programmers, IMO. Even well paid corpo programmers could fail and consciously doom project's codebase if someone would require of them that level of self control and care for details. Nobody would be able to prove them that they're making a crime for their comfort.

Forked project can just accumulate perfection in following Zsh syntax, it's a much simpler situation and it matches Open Source way of doing things.

@HaleTom

This comment has been minimized.

Copy link

commented Aug 31, 2018

I believe that the answer may depend on whether whether one shellcheck syntax-checker can inherit easily from another.

If it's possible, then perhaps something like this is possible:

  1. Move all current bash syntax checkers into bazsh checkers
  2. Create empty zsh syntax checkers inheriting from bazsh. These can be added to incrementally.
  3. Allow bug reports to discover bash-only checkers in bazsh (and therefore in zsh), and move them to bash (which also inherits from bazsh.

This would hopefully avoid much of the shell version checking you describe.

I wonder how it is currently done for ksh vs bash that @koalaman mentions here?

@koalaman could you comment?

@danielbayley

This comment has been minimized.

Copy link

commented Sep 2, 2018

Wouldn't the simplest approach be to create a zsh branch or in the initial logic choose which parser to use based on the bang line #! /bin/zsh/#! /bin/bash etc…?

@psprint

This comment has been minimized.

Copy link

commented Sep 29, 2018

@HaleTom Ok the idea seems reasonable, it opens a way to do programming not whole-picture-grasping, which is pretty much the thing that I opt for. I think at this point, what's needed, is the Spellcheck authors to tell us how much the code is done in direct-structural way vs. indirect-with-high-degree-of-freedom way. I mean, you wrote about kind of syntax-"backends" that can be moved level-up or level-down between general-shell and specific-{bash,zsh} levels, to gradually separate things, but in reality I suspect this will have to be coded first, and I wonder how much parsing-knowledge or parsing-certainty/comfort would it require to actually approach this task.

@koalaman

This comment has been minimized.

Copy link
Owner

commented Nov 30, 2018

I appreciate your assumptions that ShellCheck is an enterprise level project with smart abstractions and a clever design, but you're really overestimating both it and me. Apart from being written in Haskell, ShellCheck is very simple and straight forward.

It essentially just has one parser and one set of checks that are applied to the result.

The parser handles all syntax for all shells the same way. If you use a Bash specific feature in Dash, such as for ((;;)), then it'll be parsed and interpreted as that Bash feature, and will get all corresponding Bash warnings. There's hopefully also a check that warns that it's unsupported in Dash, but that doesn't change how ShellCheck processes it.

There's no reason why you couldn't additionally recognize Zsh for x (1 2 3) and do the same. Note especially that it's not necessary to implement more than a single statement type at a time before you can run it and see the effects.

As for the checks, they don't have groups or stages. The only structure imposed is optional, and is purely for convenience and efficiency (like registering to handle a command name instead of having to iterate the AST to look for them). If a check is specific to a different shell, it'll just check and do an early return.

I doubt any of that would be a concern though. 95%+ of checks that apply to Bash also apply to Zsh (fgrep isn't less deprecated for example). There's probably <5 that don't, and those could just return early as well.

@psprint

This comment has been minimized.

Copy link

commented Dec 7, 2018

I see, from that point of view it looks easy. With the possible problems anticipated by me, I was thinking about things that are more subtle than an additional syntax. Like for example word-splitting being the default in bash, Zsh whole array referencing with $array instead of ${array[@]} (however: it works like in bash if ksharrays option is set), array indexing starting at 0 not 1, etc. I have had an impression that there is much such things, but I cannot be sure about this, would have to sit down and think about this a little longer.

@psprint

This comment has been minimized.

Copy link

commented Dec 18, 2018

@koalaman; Can it be said, that after implementing checks for a most easily-occurring mistakes, Shellcheck took a path of implementing checks for a more complex & rare, more specific, narrow, fine-grained mistakes? And that that's why the execution of a shell script has been implemented – to be able to dig into the states of the code and catch the fine-grained mistakes?

@koalaman

This comment has been minimized.

Copy link
Owner

commented Dec 18, 2018

ShellCheck does not support execution of shell scripts. Are you thinking of shfmt?

@psprint

This comment has been minimized.

Copy link

commented Dec 18, 2018

Ah, yes, shfmt. That said, I've finally have looked at the README. It surprised me positively because it listed the checks that are being done. I think that looking at the checks I can confirm that Shellcheck went from general scripting error checking (e.g. sh-word-split problems) to fine graned checks that e.g. target a syntax of a single command, like find.

@koalaman

This comment has been minimized.

Copy link
Owner

commented Dec 18, 2018

ShellCheck's original purpose was to auto-suggest relevant Wooledge BashFAQs and Pitfalls on IRC, so that's where most of the original checks come from. These include a mix of syntactic, semantic and command-specific issues.

@psprint

This comment has been minimized.

Copy link

commented Dec 18, 2018

Ahso, so maybe Shellcheck was fine-grained & specific from the start. I've got to the idea of fine-crafted, sophisticated-situations-targetted, when I thought "What checks I could actually do? Despite shwordsplit warnings on an unquoted variable use?" (I'm thinking about starting a Zsh linter project), and this lead me to the conclusion that I can target very specific situations but with possibly large number of them, making the linter a tool that often reports errors even though each of them is rare.

It essentially just has one parser and one set of checks that are applied to the result.

I think that if Shellcheck has a very large number of checks, then it is entering a more advanced level than the quote states, it's a level of knowledge base-like tool.

@rolfbjarne

This comment has been minimized.

Copy link

commented Jun 7, 2019

zsh will become the default shell in macOS later this year, so zsh will become much more common quite fast.

@LwsBtlr

This comment has been minimized.

Copy link

commented Jun 30, 2019

It will suddenly be installed an the default shell on tens of millions of machines.

@dpwolfe

This comment has been minimized.

Copy link

commented Jul 5, 2019

zsh will become the default shell in macOS later this year, so zsh will become much more common quite fast.

It was announced that it will release this Fall, which starts September 23rd: http://osxdaily.com/2019/06/18/macos-catalina-release-dates-public-beta-final/

In the meantime, if you upgrade an existing MacBook to the 10.15 Beta, you'll get this every time you open a new terminal session:

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.

I just set the flag in my .bash_profile to silence the warning for now until I get to porting my scripts:

export BASH_SILENCE_DEPRECATION_WARNING=1
@bp88

This comment has been minimized.

Copy link

commented Jul 10, 2019

zsh will be the default shell in macOS Catalina: https://support.apple.com/en-us/HT208050

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.