Implement 'cabal sandbox shell' #1627

Closed
23Skidoo opened this Issue Dec 21, 2013 · 22 comments

Comments

Projects
None yet
5 participants
@23Skidoo
Member

23Skidoo commented Dec 21, 2013

The original design proposal for sandboxes included the sandbox shell feature, inspired by hsenv and virthualenv. The idea was that it would start a shell with a sandbox-restricted compiler and tools, so, for example, ghci would automatically use the sandboxed package DB. It'd be nice to implement it at some point. The simplest way to implement this is probably to use GHC_PACKAGE_PATH.

Related: #1621.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Dec 21, 2013

Probably also worth updating the PATH environment variables so the invoked process has access to utilities installed in the sandbox bin directory.

alevy commented Dec 21, 2013

Probably also worth updating the PATH environment variables so the invoked process has access to utilities installed in the sandbox bin directory.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Dec 21, 2013

Why a shell rather than an arbitrary executable?

alevy commented Dec 21, 2013

Why a shell rather than an arbitrary executable?

@23Skidoo

This comment has been minimized.

Show comment
Hide comment
@23Skidoo

23Skidoo Dec 22, 2013

Member

You can run arbitrarily executables from within the shell.

Member

23Skidoo commented Dec 22, 2013

You can run arbitrarily executables from within the shell.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Dec 22, 2013

There are a few downsides to this:

  1. It's just less general. If you're able to run an arbitrary executable, that executable can always be a shell. For example, what if I want to pipe something through standard in into a process running inside the sandbox? If I can run arbitrary commands (let's call that subcommand exec to differentiate), that's trivial:

    echo "Some input" | caban sandbox exec my_sandbox_specific_utility

    How do I do that with the shell though? If the shell is necessarily running something like bash, and the cabal subcommand allows me to pass arguments, I might uses bash style arguments to run a command without an interactive terminal:

    echo "Some input" | cabal sandbox shell -c "my_sandbox_specific_utility"

    But that seems over complicated. Moreover there are odd discrepancies here, like bash takes the remaining arguments after -c while csh takes only the first argument, which one will be used on machine that actually runs this?

  2. Shells tend to pull in lots of things (for example sourcing $HOME/.profile) which may be undesirable in some situations.

  3. How do we choose which shell to run? Do we use the currently running shell? $SHELL? What if there is no currently running shell? What if we're in a chroot that doesn't expose /bin/bash?

I can go on... basically it seems like a unnecessary limitation. We're going to need to set environment variables in exactly the same way anyway. We're going to need to fork and exec a process anyway. It's just a matter of whether which process is exec'ed can be read from the arguments to the subcommand. And doing that would be way more flexible. Have a sane default of executing a shell if no arguments are given and you get the best of both worlds!

alevy commented Dec 22, 2013

There are a few downsides to this:

  1. It's just less general. If you're able to run an arbitrary executable, that executable can always be a shell. For example, what if I want to pipe something through standard in into a process running inside the sandbox? If I can run arbitrary commands (let's call that subcommand exec to differentiate), that's trivial:

    echo "Some input" | caban sandbox exec my_sandbox_specific_utility

    How do I do that with the shell though? If the shell is necessarily running something like bash, and the cabal subcommand allows me to pass arguments, I might uses bash style arguments to run a command without an interactive terminal:

    echo "Some input" | cabal sandbox shell -c "my_sandbox_specific_utility"

    But that seems over complicated. Moreover there are odd discrepancies here, like bash takes the remaining arguments after -c while csh takes only the first argument, which one will be used on machine that actually runs this?

  2. Shells tend to pull in lots of things (for example sourcing $HOME/.profile) which may be undesirable in some situations.

  3. How do we choose which shell to run? Do we use the currently running shell? $SHELL? What if there is no currently running shell? What if we're in a chroot that doesn't expose /bin/bash?

I can go on... basically it seems like a unnecessary limitation. We're going to need to set environment variables in exactly the same way anyway. We're going to need to fork and exec a process anyway. It's just a matter of whether which process is exec'ed can be read from the arguments to the subcommand. And doing that would be way more flexible. Have a sane default of executing a shell if no arguments are given and you get the best of both worlds!

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Jan 10, 2014

Here's a prototype (implemented as a shell function in my zshrc) I've been using:

cabal() {
  if [[ $1 == "exec" ]]
  then
    shift
    local dir=$PWD conf db
    while :; do
      conf=$dir/cabal.sandbox.config
      [[ ! -f $conf ]] || break
      if [[ -z $dir ]]; then
          echo "Cannot find cabal.sandbox.config" >&2
          return 1
      fi
      dir=${dir%/*}
    done

    db=$(sed -ne '/^package-db: */{s///p;q;}' "$conf")
    if [[ -d $db ]]; then
      pkg_path=$(command cabal sandbox hc-pkg list 2> /dev/null | grep \: | tac | sed 's/://' | paste -d: - -)
      if [[ $# == 0 ]]; then
        GHC_PACKAGE_PATH=${pkg_path} PATH=$(dirname $db)/bin:$PATH $SHELL
      else
        GHC_PACKAGE_PATH=${pkg_path} PATH=$(dirname $db)/bin:$PATH "$@"
      fi
    fi
  else
    command cabal $@
  fi
}

It's not perfect, but works reasonably well. cabal exec [some_command] use cabal sandbox hc-pkg to configure the GHC_PACKAGE_PATH environment variable, and the sandbox path from cabal.sandbox.config to set the PATH environment variable.

alevy commented Jan 10, 2014

Here's a prototype (implemented as a shell function in my zshrc) I've been using:

cabal() {
  if [[ $1 == "exec" ]]
  then
    shift
    local dir=$PWD conf db
    while :; do
      conf=$dir/cabal.sandbox.config
      [[ ! -f $conf ]] || break
      if [[ -z $dir ]]; then
          echo "Cannot find cabal.sandbox.config" >&2
          return 1
      fi
      dir=${dir%/*}
    done

    db=$(sed -ne '/^package-db: */{s///p;q;}' "$conf")
    if [[ -d $db ]]; then
      pkg_path=$(command cabal sandbox hc-pkg list 2> /dev/null | grep \: | tac | sed 's/://' | paste -d: - -)
      if [[ $# == 0 ]]; then
        GHC_PACKAGE_PATH=${pkg_path} PATH=$(dirname $db)/bin:$PATH $SHELL
      else
        GHC_PACKAGE_PATH=${pkg_path} PATH=$(dirname $db)/bin:$PATH "$@"
      fi
    fi
  else
    command cabal $@
  fi
}

It's not perfect, but works reasonably well. cabal exec [some_command] use cabal sandbox hc-pkg to configure the GHC_PACKAGE_PATH environment variable, and the sandbox path from cabal.sandbox.config to set the PATH environment variable.

@tibbe

This comment has been minimized.

Show comment
Hide comment
@tibbe

tibbe Jan 10, 2014

Member

I think I'm convinced we want cabal exec. It is more general and works better on platforms where shell's aren't first class (i.e. Windows).

Member

tibbe commented Jan 10, 2014

I think I'm convinced we want cabal exec. It is more general and works better on platforms where shell's aren't first class (i.e. Windows).

@etrepum

This comment has been minimized.

Show comment
Hide comment
@etrepum

etrepum Jan 12, 2014

Contributor

👍

A fun use case for this would be "cabal exec ghci" which does overlap with the use case for "cabal repl" but you could use it in a sandbox that doesn't have a top-level cabal file. For example, if you wanted to play around with some library from hackage without having to create a whole project around it.

Contributor

etrepum commented Jan 12, 2014

👍

A fun use case for this would be "cabal exec ghci" which does overlap with the use case for "cabal repl" but you could use it in a sandbox that doesn't have a top-level cabal file. For example, if you wanted to play around with some library from hackage without having to create a whole project around it.

@etrepum

This comment has been minimized.

Show comment
Hide comment
@etrepum

etrepum Jan 12, 2014

Contributor

It looks like Cabal doesn't do anything with GHC_PACKAGE_PATH internally, but it's relatively straightforward to set the PATH and run something. The other tricky part is getting cabal to ignore options after exec, -- can be used but maybe that's not the best UI?

Contributor

etrepum commented Jan 12, 2014

It looks like Cabal doesn't do anything with GHC_PACKAGE_PATH internally, but it's relatively straightforward to set the PATH and run something. The other tricky part is getting cabal to ignore options after exec, -- can be used but maybe that's not the best UI?

@etrepum

This comment has been minimized.

Show comment
Hide comment
@etrepum

etrepum Jan 12, 2014

Contributor

Here's a prototype that's missing the GHC_PACKAGE_PATH support https://github.com/etrepum/cabal/tree/cabal-exec

Contributor

etrepum commented Jan 12, 2014

Here's a prototype that's missing the GHC_PACKAGE_PATH support https://github.com/etrepum/cabal/tree/cabal-exec

@23Skidoo

This comment has been minimized.

Show comment
Hide comment
@23Skidoo

23Skidoo Jan 12, 2014

Member

@etrepum

A fun use case for this would be "cabal exec ghci" which does overlap with the use case for "cabal repl" but you could use it in a sandbox that doesn't have a top-level cabal file. For example, if you wanted to play around with some library from hackage without having to create a whole project around it.

cabal repl already supports this in HEAD.

Member

23Skidoo commented Jan 12, 2014

@etrepum

A fun use case for this would be "cabal exec ghci" which does overlap with the use case for "cabal repl" but you could use it in a sandbox that doesn't have a top-level cabal file. For example, if you wanted to play around with some library from hackage without having to create a whole project around it.

cabal repl already supports this in HEAD.

@tibbe

This comment has been minimized.

Show comment
Hide comment
@tibbe

tibbe Mar 5, 2014

Member

I think this (cabal exec) would be nice to have in 1.20, which I plan to release at the end of March.

Member

tibbe commented Mar 5, 2014

I think this (cabal exec) would be nice to have in 1.20, which I plan to release at the end of March.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Mar 5, 2014

@tibbe sounds great! What is the right thing to do about other (non-GHC) compilers? @etrepum is your prototype a good starting point in your opinion? I'll pull from your repo and try to get the GHC_PACKAGE_PATH set in that as well.

alevy commented Mar 5, 2014

@tibbe sounds great! What is the right thing to do about other (non-GHC) compilers? @etrepum is your prototype a good starting point in your opinion? I'll pull from your repo and try to get the GHC_PACKAGE_PATH set in that as well.

@23Skidoo

This comment has been minimized.

Show comment
Hide comment
@23Skidoo

23Skidoo Mar 5, 2014

Member

We don't support sandboxes for non-GHC compilers anyway.

Member

23Skidoo commented Mar 5, 2014

We don't support sandboxes for non-GHC compilers anyway.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Mar 5, 2014

Oh, well that makes things easy :)

alevy commented Mar 5, 2014

Oh, well that makes things easy :)

@etrepum

This comment has been minimized.

Show comment
Hide comment
@etrepum

etrepum Mar 5, 2014

Contributor

@alevy I think it's probably a good starting point

Contributor

etrepum commented Mar 5, 2014

@alevy I think it's probably a good starting point

@benarmston

This comment has been minimized.

Show comment
Hide comment
@benarmston

benarmston Apr 6, 2014

Contributor

@alevy are you still working on this? I had a quick look at it last night and made some progress. I'm happy to finish it off and send in a PR, but I don't want to be treading on anyone's toes, so to speak.

Contributor

benarmston commented Apr 6, 2014

@alevy are you still working on this? I had a quick look at it last night and made some progress. I'm happy to finish it off and send in a PR, but I don't want to be treading on anyone's toes, so to speak.

@23Skidoo

This comment has been minimized.

Show comment
Hide comment
@23Skidoo

23Skidoo Apr 6, 2014

Member

@benarmston Feel free to go forward with this, I'll be happy to review.

Member

23Skidoo commented Apr 6, 2014

@benarmston Feel free to go forward with this, I'll be happy to review.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Apr 6, 2014

@benarmston yes please. I made a bit of progress but I won't be offended (at all!) if you go ahead with this. Just to relay my experience in case it's helpful, the blocking point I encountered is that cabal doesn't actually seem to be aware of where the global package is. Instead it relies on the GHC -global-package-db flag. The solution I was going to use, but didn't get around to, was to invoke ghc-pkg list with the options that cabal sandbox hc-pkg list uses and simply parse the output. Seems a bit hacky, but would work.

alevy commented Apr 6, 2014

@benarmston yes please. I made a bit of progress but I won't be offended (at all!) if you go ahead with this. Just to relay my experience in case it's helpful, the blocking point I encountered is that cabal doesn't actually seem to be aware of where the global package is. Instead it relies on the GHC -global-package-db flag. The solution I was going to use, but didn't get around to, was to invoke ghc-pkg list with the options that cabal sandbox hc-pkg list uses and simply parse the output. Seems a bit hacky, but would work.

@alevy

This comment has been minimized.

Show comment
Hide comment
@alevy

alevy Apr 6, 2014

Here's my branch in case it's helpful (although I think it's mostly just redoing what @etrepum had already done...):
https://github.com/alevy/cabal/tree/exec

alevy commented Apr 6, 2014

Here's my branch in case it's helpful (although I think it's mostly just redoing what @etrepum had already done...):
https://github.com/alevy/cabal/tree/exec

@benarmston

This comment has been minimized.

Show comment
Hide comment
@benarmston

benarmston Apr 6, 2014

Contributor

Thanks for the link to your branch @alevy. The lack of knowledge about the global package flag is pretty much where I'm at currently. I have a couple of ideas though.

With luck I'll have something for you to review in the next day or two, @23Skidoo.

Contributor

benarmston commented Apr 6, 2014

Thanks for the link to your branch @alevy. The lack of knowledge about the global package flag is pretty much where I'm at currently. I have a couple of ideas though.

With luck I'll have something for you to review in the next day or two, @23Skidoo.

@benarmston

This comment has been minimized.

Show comment
Hide comment
@benarmston

benarmston Apr 6, 2014

Contributor

Things went better than I thought they might. I have a working implementation although it needs some refactoring love applying to it.

Contributor

benarmston commented Apr 6, 2014

Things went better than I thought they might. I have a working implementation although it needs some refactoring love applying to it.

@benarmston

This comment has been minimized.

Show comment
Hide comment
@benarmston

benarmston Jun 28, 2014

Contributor

I believe that this can be closed.

Contributor

benarmston commented Jun 28, 2014

I believe that this can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment