add shellcheck, a static analysis tool for shell scripts #21231

mislav · 2013-07-15T18:57:49Z

The project doesn't seem to have a version, so this is dubbed v0.0.1 and is frozen to the git SHA where the current master branch is.

Problems with this formula's build process (seeking feedback):

cabal update - Necessary to run at least once after Cabal has been installed to fetch the list of available packages. However, this writes the cached files to ~/.cabal (i.e. outside of Homebrew's prefix).
cabal install ... - Also writes some files to ~/.cabal, even with --global flag enabled. This doesn't perform the checks whether the packages are already installed. Is it worth adding a Haskell package dependency checker to Homebrew for this?
Tons of things to install for just a 7.5MB resulting binary! "ghc" is ~700 MB on disk, and build time in total can be around 10-15 minutes. Maybe offer a precompiled binary instead? I'm not sure about the Homebrew process around that, or portability.

mislav · 2013-07-15T19:22:02Z

I don't know why Homebrew CI fails:

Preprocessing executable 'cabal' for cabal-install-1.16.0.2...
Distribution/Client/Types.hs:31:8:
    Could not find module `Network.URI'
    There are files missing in the `network-2.3.1.1' package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.

Worked for me without problem on OS X 10.8.4. I didn't have any Haskell development environment prior to doing brew install ghc and cabal-install.

MikeMcQuaid · 2013-07-15T20:18:23Z

Yeh, I think this probably isn't going to work as long as it requires writing stuff to .cabal. Any way of overriding that somehow?

mislav · 2013-07-15T20:26:18Z

Probably but I have to figure it out first, or someone who knows Haskell packages can chime in. You can leave this open in the meantime. Also there is the matter of the mysterious CI failure.

Permission to implement a Haskell package checker?

depends_on :haskell => 'regex-posix'

mistydemeo · 2013-07-15T20:31:05Z

If there isn't a stable release yet, we're not likely to accept this at this point. Can you ask the author about tagging a release?

MikeMcQuaid · 2013-07-15T20:31:34Z

Haskell package version checker seems like a good idea.

mislav · 2013-07-15T20:32:40Z

@mistydemeo Alright, going to do that

mislav · 2013-07-15T22:09:29Z

Here is my compiled binary: http://up.mislav.net/QF2i/shellcheck [7.5 MB]

How about we distribute that instead? Not sure about the portability of this one; don't have a Snow Leopard to test on

mistydemeo · 2013-07-15T22:12:06Z

How about we get the bots to bottle it instead, so that we a) get a binary tailored to multiple OSs, and b) users can build from source should they want it?

mislav · 2013-07-15T22:14:03Z

You had me at "How about we get the bots to"

mistydemeo · 2013-07-15T22:26:58Z

Mike and I can do that part. :D Don't worry about it. The important part is just to have a formula that can build from source on our bots.

As a part of every build process the bots build every piece of software for Snow Leopard, Lion and Mountain Lion, and generate bottles, e.g. binary packages. We don't actually upload all of those binary packages for users at the moment, but we do for software that's inconvenient to build for whatever reason. Haskell stuff definitely fits there, as far as I'm concerned.

@MikeMcQuaid If we do start bottling Haskell packages there are other popular packages I'd like to see included too, so we don't require users to go through the whole rigorous rigamarole.

mistydemeo · 2013-07-15T22:29:08Z

Library/Formula/shellcheck.rb

+  head     'https://github.com/koalaman/shellcheck.git'
+
+  depends_on 'ghc' => :build
+  depends_on 'cabal-install' => :recommended


I see you call cabal in the formula. Wouldn't that mean that this is mandatory, not just recommended?

If the person either installed Cabal themselves elsewhere on the system, or has haskell-platform from Homebrew, they already have Cabal. So I've made it optional that you may say --witout-cabal-install (at least that's what I think I did with :recommended?)

Probably better in that case to have a CabalInstalled requirement that checks if one already exists, and with a default formula to install cabal-install otherwise. You can see a few requirements set up like that here: https://github.com/mxcl/homebrew/blob/master/Library/Homebrew/requirements.rb

Yep, working on that since you've posted the initial comment :)

mislav · 2013-07-15T23:37:27Z

@mistydemeo Updated! Please have a look.

It now detects missing Haskell packages but won't install with Cabal by itself. Rather, like for RubyGems dependencies, it will fail with manual installation instructions:

shellcheck: Unsatisfied dependency: regex-compat
Homebrew does not provide Haskell dependencies; install with:
  cabal install --global regex-compat
shellcheck: Unsatisfied dependency: regex-posix
Homebrew does not provide Haskell dependencies; install with:
  cabal install --global regex-posix
Error: Unsatisifed requirements failed this build.

The user then has to run cabal update and cabal install --global regex-compat regex-posix to proceed. This will write cache files to their ~/.cabal directory but at least this formula isn't reponsible for that anymore.

adamv · 2013-07-17T15:58:06Z

May need to be rebased on master.

adamv · 2013-07-18T19:44:33Z

Can we get support for haskel deps as a separate pull request?

mislav · 2013-07-19T22:36:15Z

Can we get support for haskel deps as a separate pull request?

Done. I don't understand why you make me do this when you could have just cherry picked the commits.

adamv · 2013-07-19T23:06:52Z

Because I am lazier than you.

Avoids initializing anything in the home directory

When detecting a missing package, it suggests that you install it with: cabal --install global <PACKAGE> Example usage in formulae: depends_on 'parsec' => :haskell

Allows formulae to conditionally depend on "cabal-install" if `cabal` executable is not present. Otherwise, `cabal` might have already been provided from "haskell-platform" or by manual install. Usage: depends_on :cabal

mislav · 2013-07-24T10:03:08Z

I have updated shellcheck formula to reference the recently tagged version.

BTW, the author says:

If you're considering binary distribution, you can easily link it statically to prevent portability issues:
ghc -O9 -optl-static -optl-pthread --make shellcheck

cartazio · 2013-08-01T01:42:11Z

Library/Homebrew/requirements/language_module_dependency.rb

@@ -46,6 +47,7 @@ def command_line
      when :python3 then "pip3 install"
      when :rbx     then "rbx gem install"
      when :ruby    then "gem install"
+      when :haskell then "cabal install --global"


why is there the --global flag
here? it is really really really bad practice to force global package installs with cabal.

cartazio · 2013-08-01T01:44:30Z

to contextualize: * DO NOT PUT --global* in any build script that is doing a cabal install

It forces the end users package db to have the associated versions installed globally.

which is really hard to fix when it creates conflicting dependencies in haskell land.

considering most folks who are installing haskell via brew rather than via a ghc.pkg or haskellplatform.pkg or the like will be new haskell users, and adding packages to the global ghc pkg db should not be done automagically. Its a really really bad idea.

dysinger · 2013-08-01T01:50:29Z

--global is yucky

cartazio · 2013-08-01T02:05:24Z

see our additional remarks here: #21335

jacknagel · 2013-08-01T03:12:14Z

Just to clarify, Homebrew wouldn't ever execute this command, it's output as part of an error message when the dependency is not satisfied.

I understand your point still applies, but I don't want there to be any confusion about what that string is there for.

cartazio · 2013-08-01T05:55:24Z

umm, WHY is --global there? there is never a good reason to have "cabal install --global" rather than "cabal install" in an error message. It should NEVER be there.

cartazio · 2013-08-01T05:56:02Z

we have not objections to "cabal install", its that "cabal install --global" is NEVER a good idea.
(though i'm VERY relieved to hear that its not something that will be executed.)

cartazio · 2013-08-01T06:03:23Z

on an unrelated note: a cabal update before a cabal install run will help with resolving packages that are needed.

ghost · 2013-08-01T10:54:27Z

I think the right way to do this, which @dysinger suggested in #21335, is to create sandboxed builds of whatever haskell tools one wants to install. This can be done with cabal-dev: http://hackage.haskell.org/package/cabal-dev

Sandboxing would allow us to:

avoid touching ~/.cabal which @MikeMcQuaid was concerned about
avoid interfering with the user's existing or future haskell configuration
avoid needing the user to run cabal update, caball install, etc. manually (potentially creating other problems)

It may take longer and use more space but I think it is a cleaner solution that requires less complication for the user and potentially less support headaches for homebrew and haskell devs. However, since this approach ought to work particularly well with bottling so those concerns may be moot anyway.

ghost · 2013-08-01T11:09:00Z

Aside from the sandboxing issue, we ought to consider a general approach for handling Haskell packages people may want to install. Creating an individual formula for each one may be reasonable for now but it will mean a lot of duplication of effort. I'm thinking a better solution may be to create a 'brew cabal' tool like 'brew pip' and 'brew gem' and use that to query hackage and provide package availability and versioning information. Installing a package through 'brew cabal' could then use the sandbox approach to keep things nice and tidy.

Is there interest in this idea? Might take me awhile to get to it but I could try to give it a go.

mislav · 2013-08-01T12:14:28Z

Please continue any Haskell packages/Cabal discussion in #21335.

cabal-dev looks interesting. Anyone is welcome to use it to make an alternative to my patch. @darinmorrison: a brew-cabal command might be useful to Haskell developers but not to our need here which is to simply compile an executable with necessary dependencies. We want to avoid users doing any work. You could start maintaining brew-cabal in your own git repo and people could brew tap your repo to get the command available.

cartazio · 2013-08-01T16:40:42Z

Library/Homebrew/requirements/language_module_dependency.rb

@@ -31,6 +31,7 @@ def the_test
      when :python3 then %W{/usr/bin/env python3 -c import\ #{@import_name}}
      when :ruby then %W{/usr/bin/env ruby -rubygems -e require\ '#{@import_name}'}
      when :rbx then %W{/usr/bin/env rbx -rubygems -e require\ '#{@import_name}'}
+      when :haskell then %{/usr/bin/env ghc-pkg list '#{@import_name}' --global --simple-output --names-only | grep .}


this --global is really a bad idea

adamv · 2013-10-24T16:19:59Z

Hey everyone what's the status of including this particular piece of software?

mislav · 2013-10-24T21:21:32Z

@adamv I was told that my approach is bad and the discussion went on in #21335. There is no PR yet from others that makes this better, to my knowledge. We were waiting on the release of the mystical cabal-dev.

Closing this since there is incentive to solve this in a better way. Ideally, the shellcheck formula would install pre-built binaries.

cartazio · 2013-10-25T03:46:01Z

its not cabal-dev, cabal-install 1.18,

the cabal-install 1.18 formulae is out.

the folks who volunteered to help are kinda busy dealing with the general Mavericks support issue and getting the GHC hotfix sorted out. Once the dust settles on that, they'll probably be happy to help. They're kinda busy rightnow

MikeMcQuaid · 2013-10-25T10:12:49Z

@mislav it was solved in a non-general way in the cabal-install formula. I tried to adapt pandoc to do the same but ran against some upstream issues. It should work for shellcheck though. See my WIP here:
https://github.com/mikemcquaid/homebrew/commit/47312a

MikeMcQuaid · 2013-10-28T09:03:58Z

@darinmorrison Sounds good. Will be good to be able to drop the apple-gcc42 dependency. I didn't think you'd beat Go to it, though ;)

mislav mentioned this pull request Jul 15, 2013

Tag a stable release koalaman/shellcheck#3

Closed

mistydemeo reviewed Jul 15, 2013
View reviewed changes

mislav mentioned this pull request Jul 19, 2013

Cabal special dependency + Haskell package support #21335

Closed

mistydemeo mentioned this pull request Jul 21, 2013

pandoc: 1.11.1 (new formula) #21373

Closed

mislav added 4 commits July 24, 2013 11:48

install cabal-install in "global" as opposed to "user" mode

567e467

Avoids initializing anything in the home directory

add :haskell dependency type that checks ghc-pkg

a4eb08c

When detecting a missing package, it suggests that you install it with: cabal --install global <PACKAGE> Example usage in formulae: depends_on 'parsec' => :haskell

add :cabal special dependency

8267421

Allows formulae to conditionally depend on "cabal-install" if `cabal` executable is not present. Otherwise, `cabal` might have already been provided from "haskell-platform" or by manual install. Usage: depends_on :cabal

add shellcheck, a static analysis tool for shell scripts

f9f1761

cartazio reviewed Aug 1, 2013
View reviewed changes

mislav closed this Oct 24, 2013

parkr mentioned this pull request Oct 31, 2013

Add shellcheck v0.2.0. #23839

Closed

NotSqrt mentioned this pull request Jan 17, 2014

brew install not working SublimeLinter/SublimeLinter-shellcheck#2

Closed

michaelsanford mentioned this pull request Jan 23, 2014

Added cabal setup instructions for Mac OS X to README koalaman/shellcheck#67

Merged

Homebrew locked and limited conversation to collaborators Feb 17, 2016

add shellcheck, a static analysis tool for shell scripts #21231

add shellcheck, a static analysis tool for shell scripts #21231

Conversation

mislav commented Jul 15, 2013

mislav commented Jul 15, 2013

MikeMcQuaid commented Jul 15, 2013

mislav commented Jul 15, 2013

mistydemeo commented Jul 15, 2013

MikeMcQuaid commented Jul 15, 2013

mislav commented Jul 15, 2013

mislav commented Jul 15, 2013

mistydemeo commented Jul 15, 2013

mislav commented Jul 15, 2013

mistydemeo commented Jul 15, 2013

mistydemeo Jul 15, 2013

Choose a reason for hiding this comment

mislav Jul 15, 2013

Choose a reason for hiding this comment

mistydemeo Jul 15, 2013

Choose a reason for hiding this comment

mislav Jul 15, 2013

Choose a reason for hiding this comment

mislav commented Jul 15, 2013

adamv commented Jul 17, 2013

adamv commented Jul 18, 2013

mislav commented Jul 19, 2013

adamv commented Jul 19, 2013

mislav commented Jul 24, 2013

cartazio Aug 1, 2013

Choose a reason for hiding this comment

cartazio commented Aug 1, 2013

dysinger commented Aug 1, 2013

cartazio commented Aug 1, 2013

jacknagel commented Aug 1, 2013

cartazio commented Aug 1, 2013

cartazio commented Aug 1, 2013

cartazio commented Aug 1, 2013

ghost commented Aug 1, 2013

ghost commented Aug 1, 2013

mislav commented Aug 1, 2013

cartazio Aug 1, 2013

Choose a reason for hiding this comment

adamv commented Oct 24, 2013

mislav commented Oct 24, 2013

cartazio commented Oct 25, 2013

MikeMcQuaid commented Oct 25, 2013

MikeMcQuaid commented Oct 28, 2013