Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cabal v2-repl scriptfile.hs #6149

Closed
phadej opened this issue Jul 21, 2019 · 9 comments · Fixed by #7851
Closed

cabal v2-repl scriptfile.hs #6149

phadej opened this issue Jul 21, 2019 · 9 comments · Fixed by #7851

Comments

@phadej
Copy link
Collaborator

phadej commented Jul 21, 2019

cabal v2-repl scriptfile.hs outside project to load a scriptfile.hs into repl (GHCi).


Suppose I have a script file like:

{- cabal:
build-depends: base, splitmix
-}

import Data.List (unfoldr)
import Data.Word (Word64)
import System.Random.SplitMix (nextWord64, mkSMGen)

numbers :: [Word64]
numbers = unfoldr (Just . nextWord64) (mkSMGen 42)

main :: IO ()
main = print $ take 10 numbers

cabal v2-run -v0 scriptfile.hs works perfectly, however it takes quite a lot of time, 10s on my machine!

% time cabal v2-run -v0 ExRun.hs
[1275548033995301424,10417309031967933079,2112719111588962399,2726445820918627087,4609004260007739600,11074269206337254866,9733109294540808081,6450629194222476823,8084222243108663947,16253064923614931039]
cabal v2-run -v0 ExRun.hs  10,70s user 1,01s system 99% cpu 11,724 total

Slowness is one issue, after all there's compiling involved. It however makes development of scriptfiles extremely painful.

Yet, if the following worked, the development would be a lot nicer (repl ftw):

% cabal v2-repl ExRun.hs
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - fake-package-0 (exe:script) (first run)
Configuring executable 'script' for fake-package-0..

...

GHCi, version 8.6.5: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Main             ( ExRun.hs, interpreted )
Ok, one module loaded.
*Main> 

*Main> main
[1275548033995301424,10417309031967933079,2112719111588962399,2726445820918627087,4609004260007739600,11074269206337254866,9733109294540808081,6450629194222476823,8084222243108663947,16253064923614931039]

worked, it would be super great.


For now, I try to cabal install --lib the dependencies, and use raw ghci.

@mpickering
Copy link
Collaborator

I'm interested in this feature. Could you give some pointers about how hard it would be to implement? (cc @fendor)

@fendor
Copy link
Collaborator

fendor commented Apr 30, 2020

I intend to work on this.
The implementation of the scriptfile feature seems to be mainly in CmdInstall. It sets up a fake project by essentially writing a *.cabal file to a temporary location and then building this fake project.

First, we should move implementation details from CmdInstall to its own module, proposing: CmdScript.hs.
The implementation itself seems trivial (we will see), the only difficulty I anticipate is to detect changes in the meta-data section of the script on reload (e.g. :r in the ghci session).

@phadej
Copy link
Collaborator Author

phadej commented Apr 30, 2020

The fake package stuff is better put into Distribution.Client.FakePackage something like that. CmdScript would be confusing as there could be cabal script functionality (but it's part of run).

@phadej
Copy link
Collaborator Author

phadej commented Apr 30, 2020

It's worth to make clear to one self how cabal v2-build, cabal v2-run and cabal v2-repl behave outside and inside project. Perfectly these won't have different behaviors / cli interfaces, and I think the easiest way to get there is to abstract common parts more heavily.

@fendor
Copy link
Collaborator

fendor commented Apr 30, 2020

so, the first implementation was actually rather trivial.
The current mechanism is to copy the file to some /tmp/cabal-repl. directory and generating a very basic fake-package.cabal. So, we can just open a repl session in that project context. (Although, admittedly, it doesnt make sense to set the cwd to /tmp/cabal-repl. but that is a task for later).
The hard problem is to use :r in the repl session. I would expect that this re-compiles changes, such as source changes and changes to the meta-data.
However, since the source file is copied, neither of these changes can be detected. I had the following ideas:

  • Symlink Script.hs to /tmp/cabal-repl.*/Main.hs
    • doesnt reliably work on windows
    • propagates updates the source file
    • doesnt work to update meta-data
  • Only write cabal.project and fake-package.cabal to /tmp/cabal-repl.*/
    Now use semantically the invocation cabal repl Script.hs --project-file /tmp/cabal-repl.*/cabal.project
    • propagates updates from the source file
    • still doesnt work for updating meta-data

Are there more options? Can we hook somehow into the reload command of the repl-session to automatically get notified about changes to Script.hs ?
Btw, stack manages to do both, on :r changes to the source file and meta-data are reloaded appropriately.


It's worth to make clear to one self how cabal v2-build, cabal v2-run and cabal v2-repl behave outside and inside project.

they are quite similar indeed!

@phadej
Copy link
Collaborator Author

phadej commented May 1, 2020

It is ok to not support updating on meta-data change. cabal repl doesn't update anything when you update .cabal file either. Changing meta-data might change everything in GHCi setup. In other word: let's not try hook anything to :r command. I say that's out-of-scope.

One way to approach this is maybe to make cabal repl Script.hs to load current project and the Script.hs first. Then no-project case (this issue) would be cabal repl /absolute/path/to/Script.hs in the fake project context.

@fendor
Copy link
Collaborator

fendor commented May 1, 2020

So, workflow is roughly as follows cabal repl Script.hs:

  • Create a fake-package.cabal with the build-depends and main-is: Main.hs.
  • create ghci-script with the content:
    :cd <cwd>
    :add <cws>/Script.hs
    
  • Loading this project. This will compile Main.hs as well as <cwd>/Script.hs and add Script.hs to the repl and set the working dir correctly. Changes to Script.hs are detected on :r.

What I dont like about this is that two files are loaded, Main.hs and <cwd>/Script.hs. It seems I cant set main-is: <cwd>/Script.hs, according to the docs, and I admit that I prefer it that way.
However, then we have to set some main-is and load this file on cabal repl Script.hs.

A work-around I have been thinking about:

  • Create library stanza instead of executable in fake-package.cabal.
    • The meta-data part is parsed by executableGrammar, so that does not really work
    • Using a different stanza when opening a repl for the script is counter-intuitive since there is a different support of fields.

Any other good way of handling this?

@neongreen
Copy link

This would be absolutely lovely to have.

@georgefst
Copy link

This would be absolutely lovely to have.

It really would.

Just to point out, seeing as it hasn't been explicitly linked to this issue, that due to #6999, a .ghc.environment file gets created in $TMP when a cabal script is run, but it's swiftly deleted. Putting that file alongside the script would otherwise allow us to open a repl with the correct environment by just running ghci.

bacchanalia added a commit to bacchanalia/cabal that referenced this issue Dec 2, 2021
repl starts in the correct directory and points directly to rather than
a dummy, so that reloading works properly.

There is a downside to the current approach which is that it uses a
different fake-project.cabal file from run and build, so it cannot share
the same cache with them.

WIP: haskell#7842
WIP: haskell#6149
@jneira jneira linked a pull request Dec 3, 2021 that will close this issue
bacchanalia added a commit to bacchanalia/cabal that referenced this issue Dec 7, 2021
bacchanalia added a commit to bacchanalia/cabal that referenced this issue Dec 13, 2021
repl starts in the correct directory and points directly to rather than
a dummy, so that reloading works properly.

There is a downside to the current approach which is that it uses a
different fake-project.cabal file from run and build, so it cannot share
the same cache with them.

WIP: haskell#7842
WIP: haskell#6149
bacchanalia added a commit to bacchanalia/cabal that referenced this issue Dec 13, 2021
bacchanalia added a commit to bacchanalia/cabal that referenced this issue Dec 23, 2021
repl starts in the correct directory and points directly to rather than
a dummy, so that reloading works properly.

There is a downside to the current approach which is that it uses a
different fake-project.cabal file from run and build, so it cannot share
the same cache with them.

WIP: haskell#7842
WIP: haskell#6149
bacchanalia added a commit to bacchanalia/cabal that referenced this issue Dec 23, 2021
jneira pushed a commit to bacchanalia/cabal that referenced this issue Dec 31, 2021
repl starts in the correct directory and points directly to rather than
a dummy, so that reloading works properly.

There is a downside to the current approach which is that it uses a
different fake-project.cabal file from run and build, so it cannot share
the same cache with them.

WIP: haskell#7842
WIP: haskell#6149
jneira pushed a commit to bacchanalia/cabal that referenced this issue Dec 31, 2021
@mergify mergify bot closed this as completed in #7851 Dec 31, 2021
mergify bot pushed a commit that referenced this issue Dec 31, 2021
* Add support for script build caching to cabal run

Enable caching of script builds by changing the location of the fake
package directory from a tmp directory to:
<cabal_dir>/scipt-builds/abs/path/to/script/

Resolves: #6354
WIP: #7842

* Add support for scripts to cabal build.

Added module Distribution.Client.ScriptUtils for code to deal with
scripts that is common between commands.

WIP: #7842

* Add script support to cabal clean.

This changes the behaviour of cabal clean to accept extra args, which it
now interprets as script files. The behaviour of cabal clean is the same
when given extra args. When given extra args it instead removes the
caches for those scripts and also any orphaned caches (caches for which
the script no longer exists)

In addition this commit changes the cache to use hashes of paths because
this significantly simplifies the implementation of clean, and more
importantly it prevents collisions when a script has the name of the
subdirectory of a previously cached script.

WIP: #7842

* Add script support to cabal repl

repl starts in the correct directory and points directly to rather than
a dummy, so that reloading works properly.

There is a downside to the current approach which is that it uses a
different fake-project.cabal file from run and build, so it cannot share
the same cache with them.

WIP: #7842
WIP: #6149

* Added changelog for pr #7851

* Fix `cabal run script.hs` issue with --builddir

Fixes tests:
cabal-testsuite/PackageTests/NewBuild/CmdRun/Script/cabal.test.hs
cabal-testsuite/PackageTests/NewBuild/CmdRun/ScriptLiterate/cabal.test.hs

* Fixes for `build script` and `repl script`

- Fix build issue introduced in 079c5f0, where build was being passed
the wrong target filter
- Fix repl issue where script didn't work in a project context.
- Refactor code to share logic between repl and build/run
- Ensure temp directories are only created when needed

* Bug fixes relating to script support

ScriptUtils:
- Hash prefix for cache dirs was applied incorrectly.
- Overwriting fake-package files causes repeated work in some cases.
CmdClean:
- Clean distdir for script when --builddir is passed
- Always clean orphans because because there is no good way to specify
they should be cleaned. This may be bad behaviour in some obscure cases
(a cache is temporarily orphaned and an unrelated clean is run), but at
worst results in a cache rebuild.

* Add tests for improved script support

- Basic script support for build/repl/clean which checks for cached
project files
- Add check for cached project files to basic run script test
- No repeated work for build/build, build/run, run/run, and repl/repl
- Clean does not remove cache for existing scripts
- Clean does remove orphaned script caches

* Fix clean bug uncovered by 5fad121

- clean was trying to read source-builds even if it didn't exist
- add test specific to this case with other clean tests

* Update documentation for better script support

Ready for review: #7851
May close: #7842, #6354, #6149

* Attempt to fix `repl script` on Windows

PR #7851

* Attempt to fix remote test failures

Test logs showed that the failures where because the tests depended on a
module from cabal-install that some ghc versions could not find.

Instead of depending on cabal-install, I copied the needed function into
Test.Cabal.Prelude (It seemed like an acceptable place for it)

PR #7851

* Attempt to fix `repl script` on Windows

PR #7851

* Attempt to fix tests on old ghc versions

Tests failing on pre-AMP ghcs due to unsanctioned use of (<$>)

PR #7851

* Feedback: Update docs and formatting

PR #7851

* Feedback: code style changes

- remove partial selectors
- make a constant for fake-package.cabal

PR #7851

* Feedback: make hidden control flow explicit

PR #7851

* Feedback: add expected fail script run tests

PR #7851

* Fix `repl script` when cwd is deeper than cachedir

PR #7851

* Use script in-place for build or run

- Set the hs-source-dir to the location of the script for build and run,
  the same as with repl
- This removes the need to copy the script
- repl no longer needs a separate cache because all three commands
  use identical project files
- Adds multi-module support to scripts for free (#6787)
- Add new build/repl test and run multi-module test

PR #7851

* Fix file-locking issue on Windows

PR #7851

* Fix script recompilation based on cwd

- Pass info about cwd to repl through --repl-options instead of hacking
  it into the project file.
- Improve paths output by makeRelativeCanonical, makeRelativeToDir, and
  makeRelativeToCwd.
- Script multi-module support works, but with warning in repl.
- Remove script multi-module mention support in docs.

PR #7851

* Make `repl script` respect --repl-no-load

* Feedback: minor refactor

Move argument truncation from targetStrings out of
withScriptContextAndSelectors to runAction

PR #7851

* Feedback: refactor and comments for repl options

PR #7851

* Don't use hs-source-dirs for scripts.

- instead pass absolute path to script in main-is
- resolves issue with relative paths on Windows
- simplifies code and gives prettier build output
- update tests because build output has changed
- removes ability to use multi-module scripts
  (which was never officially endorsed)
- remove test for multi-module scripts
- add checks for unsupported fields in scripts

PR #7851

* Update changelog for PR #7851
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants