Installing executables (was: yesod-bin as "extra build tool") #153

Closed
chreekat opened this Issue Jun 1, 2015 · 16 comments

Comments

Projects
None yet
4 participants
@chreekat
Contributor

chreekat commented Jun 1, 2015

Is yesod-bin an "extra build tool"?

From the FAQ:

How do I get extra build tools?

stack will automatically install build tools required by your packages [emphasis mine] or their dependencies, in particular alex and happy.

yesod-bin is not technically required, as anyone can use Yesod libraries without bothering with yesod-bin. However, I want it to be available to my project's contributors with a minimum of fuss. Should stack solve this problem for me?

Self-answer: Yes. I can add yesod-bin as an extra-dep, and then 'stack exec yesod' works. Still, the difference between yesod-bin and e.g. happy are a bit magical to me. Do you just have a list of packages known to need a particular tool? What distinguishes them from tool XYZ, needed to build XYZmabob? Will I still need another build tool on top of stack to solve the general problem? Why don't I have to add 'happy' as an extra-dep?

The following issue seems sufficiently related:

yesod-bin as extra-dep

After I add yesod-bin to extra-deps and rebuild, the following occurs:

$ stack exec yesod devel
Yesod devel server. Press ENTER to quit
Application can be accessed at:

http://127.0.0.1:3000
https://127.0.0.1:3443
If you wish to test https capabilities, you should set the following variable:
  export APPROOT=https://127.0.0.1:3443

WARNING: the following source files are not listed in exposed-modules or other-modules:
./Handler/About.hs
./SnowdriftEmailDaemon.hs
./SnowdriftSendmail.hs
yesod: cabal: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 2, 2015

Contributor

It's not an extra build tool in the sense of the others, since it's not used during the build process in order to compile your code. alex and happy are dependencies of the build, whereas yesod-bin is a user-facing tool. I can add that distinction to the README if it's clear (or, if you'd like, please modify the README yourself for clarity).

Now actually using yesod devel with stack's sandboxes would require some changes to yesod-bin, since yesod-bin depends on cabal to do the build. That's a change I'll happily make in the future :)

Contributor

snoyberg commented Jun 2, 2015

It's not an extra build tool in the sense of the others, since it's not used during the build process in order to compile your code. alex and happy are dependencies of the build, whereas yesod-bin is a user-facing tool. I can add that distinction to the README if it's clear (or, if you'd like, please modify the README yourself for clarity).

Now actually using yesod devel with stack's sandboxes would require some changes to yesod-bin, since yesod-bin depends on cabal to do the build. That's a change I'll happily make in the future :)

@chreekat

This comment has been minimized.

Show comment
Hide comment
@chreekat

chreekat Jun 2, 2015

Contributor

I felt a twinge writing this ticket because it wasn't very focused, so I'm glad we agree on the definition of a build tool. This might not be the place to talk about it, but I guess the underlying question is, is it within stack's scope to set up (Haskell) binaries where a user can get to them? I want to say no, because the two tasks seem very different, yet the two tasks share many identical pain points that stack is designed to solve.

Contributor

chreekat commented Jun 2, 2015

I felt a twinge writing this ticket because it wasn't very focused, so I'm glad we agree on the definition of a build tool. This might not be the place to talk about it, but I guess the underlying question is, is it within stack's scope to set up (Haskell) binaries where a user can get to them? I want to say no, because the two tasks seem very different, yet the two tasks share many identical pain points that stack is designed to solve.

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 2, 2015

Contributor

So firstly, we have part of a solution with the stack deps command. Running stack deps yesod-bin should build the executable. The question is how do you get stack to copy that executable to somewhere you can use it?

I was talking about the lack of an install command with @feuerbach yesterday. Ideally, stack install yesod-bin would build yesod-bin and then copy the resulting executable to some "global"-ish install place. There are two problems:

  1. Where would that location be? We could do the cabal thing and copy to ~/.stack/bin, which is reasonable, but then we'd need to tell users to modify their PATH to include that directory (which, again, may be perfectly reasonable).
  2. install as a command comes with a lot of mental baggage from the cabal world, where install means "grab dependencies, install them to a global place, build the local package, and copy it over." We do dependencies building automatically in stack, but keep it all isolated. There's no concept of "installing" libraries, only executables. I'm concerned that people will be very confused with an install command.

Perhaps I'm overthinking this and we should just provide install, or perhaps we should come up with some other command. I'm open to thoughts. In any event, for now, I've been using the following incantation while working on stack:

stack build && cp $(stack exec which stack) ~/.cabal/bin
Contributor

snoyberg commented Jun 2, 2015

So firstly, we have part of a solution with the stack deps command. Running stack deps yesod-bin should build the executable. The question is how do you get stack to copy that executable to somewhere you can use it?

I was talking about the lack of an install command with @feuerbach yesterday. Ideally, stack install yesod-bin would build yesod-bin and then copy the resulting executable to some "global"-ish install place. There are two problems:

  1. Where would that location be? We could do the cabal thing and copy to ~/.stack/bin, which is reasonable, but then we'd need to tell users to modify their PATH to include that directory (which, again, may be perfectly reasonable).
  2. install as a command comes with a lot of mental baggage from the cabal world, where install means "grab dependencies, install them to a global place, build the local package, and copy it over." We do dependencies building automatically in stack, but keep it all isolated. There's no concept of "installing" libraries, only executables. I'm concerned that people will be very confused with an install command.

Perhaps I'm overthinking this and we should just provide install, or perhaps we should come up with some other command. I'm open to thoughts. In any event, for now, I've been using the following incantation while working on stack:

stack build && cp $(stack exec which stack) ~/.cabal/bin
@chreekat

This comment has been minimized.

Show comment
Hide comment
@chreekat

chreekat Jun 2, 2015

Contributor

I would say you use a completely different name precisely because so much of the confusion around cabal stems from the fact that 'cabal install' and 'cabal build' are such different commands. Better to avoid that legacy.

The tradeoff, of course, would be having two separate commands.

That would be acceptable to me, but if it's too much, perhaps one could use 'install-tool' instead of 'install'?

As for where to install, I would suggest ~/.local/bin. It's a can of worms, but npm uses it, and apparently python too. Fedora supports it out of the box, but I had to add it to my PATH manually on Ubuntu.

Using ~/.local/bin would be a nice shortcut for me because I already sandbox-build tools and copy them there manually. :)

Contributor

chreekat commented Jun 2, 2015

I would say you use a completely different name precisely because so much of the confusion around cabal stems from the fact that 'cabal install' and 'cabal build' are such different commands. Better to avoid that legacy.

The tradeoff, of course, would be having two separate commands.

That would be acceptable to me, but if it's too much, perhaps one could use 'install-tool' instead of 'install'?

As for where to install, I would suggest ~/.local/bin. It's a can of worms, but npm uses it, and apparently python too. Fedora supports it out of the box, but I had to add it to my PATH manually on Ubuntu.

Using ~/.local/bin would be a nice shortcut for me because I already sandbox-build tools and copy them there manually. :)

@snoyberg snoyberg self-assigned this Jun 2, 2015

@snoyberg snoyberg added this to the Second release milestone Jun 2, 2015

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 3, 2015

Contributor

So to summarize, I'm seeing the following work items coming out of this:

  1. Add a new install-tool command (name to be bikeshedded to death) that will do the same thing as build and then copy any executables generated by it to the user bin directory. We'll use ~/.local/bin on POSIX systems, and figure something out for Windows. We should probably warn the user if that directory is not on his/her PATH.
  2. Update the README to include information on this new install-tool command and explain how it's different from cabal install.

Does that sound reasonable?

Contributor

snoyberg commented Jun 3, 2015

So to summarize, I'm seeing the following work items coming out of this:

  1. Add a new install-tool command (name to be bikeshedded to death) that will do the same thing as build and then copy any executables generated by it to the user bin directory. We'll use ~/.local/bin on POSIX systems, and figure something out for Windows. We should probably warn the user if that directory is not on his/her PATH.
  2. Update the README to include information on this new install-tool command and explain how it's different from cabal install.

Does that sound reasonable?

@snoyberg snoyberg changed the title from yesod-bin as "extra build tool" to Installing executables WAS yesod-bin as "extra build tool" Jun 3, 2015

@snoyberg snoyberg modified the milestones: Third release, Second release Jun 3, 2015

@feuerbach

This comment has been minimized.

Show comment
Hide comment
@feuerbach

feuerbach Jun 3, 2015

Contributor

In addition to copying, install-tool should also do stripping. That can reduce the size of a Haskell executable by ~40%.

Contributor

feuerbach commented Jun 3, 2015

In addition to copying, install-tool should also do stripping. That can reduce the size of a Haskell executable by ~40%.

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 3, 2015

Contributor

👍

Contributor

snoyberg commented Jun 3, 2015

👍

@mboes

This comment has been minimized.

Show comment
Hide comment
@mboes

mboes Jun 3, 2015

Contributor
Contributor

mboes commented Jun 3, 2015

@snoyberg snoyberg added the ready label Jun 9, 2015

@chreekat chreekat changed the title from Installing executables WAS yesod-bin as "extra build tool" to Installing executables (was: yesod-bin as "extra build tool") Jun 9, 2015

@snoyberg snoyberg added in progress and removed ready labels Jun 10, 2015

snoyberg added a commit that referenced this issue Jun 10, 2015

@snoyberg snoyberg assigned chreekat and unassigned snoyberg Jun 10, 2015

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 10, 2015

Contributor

This is now implemented. Some details:

  • Running stack install-exe will install executables for the "wanted" targets, following the same rules as stack test, for example
  • It gives a warning when the destination directory is not on the PATH
  • On Windows, this cannot be used to upgrade stack yet, since we need to do some fancy footwork about moving the exe around (TODO added)

Now that I've implemented this, I'm beginning to think I made a mistake about saying we should give it a name besides install. Just because cabal has weird nomenclature doesn't mean we should do. What does everyone think about renaming the command to install instead?

Contributor

snoyberg commented Jun 10, 2015

This is now implemented. Some details:

  • Running stack install-exe will install executables for the "wanted" targets, following the same rules as stack test, for example
  • It gives a warning when the destination directory is not on the PATH
  • On Windows, this cannot be used to upgrade stack yet, since we need to do some fancy footwork about moving the exe around (TODO added)

Now that I've implemented this, I'm beginning to think I made a mistake about saying we should give it a name besides install. Just because cabal has weird nomenclature doesn't mean we should do. What does everyone think about renaming the command to install instead?

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 10, 2015

Contributor

Fixing the Windows upgrade problem was easy, and I've implemented it. But I forgot to mention: there's some discussion warranted around the path on Windows. Currently, it's using directories like C:\Users\Michael\AppData\Roaming\local\bin. Any better ideas?

Contributor

snoyberg commented Jun 10, 2015

Fixing the Windows upgrade problem was easy, and I've implemented it. But I forgot to mention: there's some discussion warranted around the path on Windows. Currently, it's using directories like C:\Users\Michael\AppData\Roaming\local\bin. Any better ideas?

@feuerbach

This comment has been minimized.

Show comment
Hide comment
@feuerbach

feuerbach Jun 10, 2015

Contributor

I have no problem with install.

Does it work for non-local targets? E.g. I may want to just say stack install pandoc.

Contributor

feuerbach commented Jun 10, 2015

I have no problem with install.

Does it work for non-local targets? E.g. I may want to just say stack install pandoc.

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 10, 2015

Contributor

Yes, it should. One thing we could consider is figuring out good behavior for running stack install outside of a project directory, presumably working the same way stack exec will end up working.

Contributor

snoyberg commented Jun 10, 2015

Yes, it should. One thing we could consider is figuring out good behavior for running stack install outside of a project directory, presumably working the same way stack exec will end up working.

snoyberg added a commit that referenced this issue Jun 10, 2015

@chreekat

This comment has been minimized.

Show comment
Hide comment
@chreekat

chreekat Jun 10, 2015

Contributor

My real preference is to have two commands with the same distinction as
exists between 'apt-get install' and 'make install'. The latter makes no
sense in a non-project directory, and the former doesn't care about $PWD at
all.

If a command is to do both, it may as well be called install. :P Maybe
we'll advance to a bright future where I can just apt-get install the
latest binaries from LTS Stackage anyway...

On Wed, Jun 10, 2015 at 4:47 AM, Michael Snoyman notifications@github.com
wrote:

Yes, it should. One thing we could consider is figuring out good behavior
for running stack install outside of a project directory, presumably
working the same way stack exec will end up working.


Reply to this email directly or view it on GitHub
#153 (comment)
.

Contributor

chreekat commented Jun 10, 2015

My real preference is to have two commands with the same distinction as
exists between 'apt-get install' and 'make install'. The latter makes no
sense in a non-project directory, and the former doesn't care about $PWD at
all.

If a command is to do both, it may as well be called install. :P Maybe
we'll advance to a bright future where I can just apt-get install the
latest binaries from LTS Stackage anyway...

On Wed, Jun 10, 2015 at 4:47 AM, Michael Snoyman notifications@github.com
wrote:

Yes, it should. One thing we could consider is figuring out good behavior
for running stack install outside of a project directory, presumably
working the same way stack exec will end up working.


Reply to this email directly or view it on GitHub
#153 (comment)
.

@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 10, 2015

Contributor

Right now, if you run stack install pandoc outside of a project directory, you'll be guaranteed to get the version from upstream, which may be pretty close to what you're looking for

Contributor

snoyberg commented Jun 10, 2015

Right now, if you run stack install pandoc outside of a project directory, you'll be guaranteed to get the version from upstream, which may be pretty close to what you're looking for

@snoyberg snoyberg closed this Jun 10, 2015

@snoyberg snoyberg removed the in progress label Jun 10, 2015

@chreekat

This comment has been minimized.

Show comment
Hide comment
@chreekat

chreekat Jun 10, 2015

Contributor

Aye, but if I run stack install pandoc inside a project, it.... oh. Hm! It does a smart thing. I am satisfied.

Well, except for the following:

Couldn't find executable make-pandoc-man-pages in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Copying from /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/pandoc to /home/b/.local/bin/pandoc
Couldn't find executable trypandoc in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Contributor

chreekat commented Jun 10, 2015

Aye, but if I run stack install pandoc inside a project, it.... oh. Hm! It does a smart thing. I am satisfied.

Well, except for the following:

Couldn't find executable make-pandoc-man-pages in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Copying from /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/pandoc to /home/b/.local/bin/pandoc
Couldn't find executable trypandoc in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
@snoyberg

This comment has been minimized.

Show comment
Hide comment
@snoyberg

snoyberg Jun 10, 2015

Contributor

That might be that the buildable flag isn't being tracked properly right
now. There's a todo in the codebase to track that.

On Wed, Jun 10, 2015, 5:44 PM Bryan Richter notifications@github.com
wrote:

Aye, but if I run stack install pandoc inside a project, it.... oh. Hm!
It does a smart thing. I am satisfied.

Well, except for the following:

Couldn't find executable make-pandoc-man-pages in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Copying from /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/pandoc to /home/b/.local/bin/pandoc
Couldn't find executable trypandoc in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/


Reply to this email directly or view it on GitHub
#153 (comment)
.

Contributor

snoyberg commented Jun 10, 2015

That might be that the buildable flag isn't being tracked properly right
now. There's a todo in the codebase to track that.

On Wed, Jun 10, 2015, 5:44 PM Bryan Richter notifications@github.com
wrote:

Aye, but if I run stack install pandoc inside a project, it.... oh. Hm!
It does a smart thing. I am satisfied.

Well, except for the following:

Couldn't find executable make-pandoc-man-pages in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/
Copying from /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/pandoc to /home/b/.local/bin/pandoc
Couldn't find executable trypandoc in directory /home/b/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/bin/


Reply to this email directly or view it on GitHub
#153 (comment)
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment