New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

file-embed makes building more fragile ? #18

Closed
simonmichael opened this Issue Apr 20, 2016 · 11 comments

Comments

Projects
None yet
4 participants
@simonmichael

simonmichael commented Apr 20, 2016

I love file-embed, but I'm not sure how to avoid file path-related breakage if I use it. I've embedded some docs in hledger. I'm seeing two problems:

  • I can no longer run ghci or stack ghci from the top of a multi-package project, eg stack ghci PKG; I have to do cd PKG; stack ghci PKG. This is not the end of the world, but I wonder if it can be avoided.
  • The travis build breaks: https://travis-ci.org/simonmichael/hledger/builds/124363146 . The command (stack +RTS -N2 -RTS build --test --haddock) works when I run it locally. I'm not sure what's going wrong here.
@snoyberg

This comment has been minimized.

Owner

snoyberg commented Apr 20, 2016

I also don't know what's going on with the Travis failure, I haven't seen that kind of thing before, and I've used file-embed extensively in Travis and other CI-built projects.

As to point 1: I've actually given this some thought over the past few months for the same reasons you raise. My thought is that we could add functions that would make the embedded paths relative to the base of the package. The way this would work is:

  • Use qLocation to get a Loc value
  • Use loc_filename to get the relative path to the source file in question
  • Ascend through parent directories until it finds a .cabal file
  • Make all paths relative to that

That shouldn't be hard to implement, and I think will be reliable for most use cases. Thoughts?

@mgsloan

This comment has been minimized.

Contributor

mgsloan commented Apr 20, 2016

@snoyberg It's a good idea, I have a module of utilities that do this, that should probably be released as a library.

@simonmichael

This comment has been minimized.

simonmichael commented Apr 20, 2016

I think I've figured out the travis issue - I had forgotten to commit some files - but weirdly it's still failing. But only on the new .info file, not on the .1 or .txt files which work fine.

@simonmichael

This comment has been minimized.

simonmichael commented Apr 20, 2016

Travis issue resolved (wildcards don't work in data-files there).

@snoyberg

This comment has been minimized.

Owner

snoyberg commented Apr 21, 2016

@simonmichael I've pushed a new branch 18-make-relative that adds a function makeRelativeToProject which should solve this problem (together with an example in the Haddocks). Can you give it a shot?

@simonmichael

This comment has been minimized.

simonmichael commented Apr 21, 2016

That works just fine. Thanks @snoyberg, @mgsloan !

@snoyberg snoyberg closed this in abcf901 Apr 21, 2016

@snoyberg

This comment has been minimized.

Owner

snoyberg commented Apr 21, 2016

Awesome, released to Hackage. I'm probably going to be rolling out similar
changes to shakespeare and yesod-core over the next few weeks.

On Thu, Apr 21, 2016 at 5:33 PM, Simon Michael notifications@github.com
wrote:

That works just fine. Thanks @snoyberg https://github.com/snoyberg!


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#18 (comment)

simonmichael added a commit to simonmichael/hledger that referenced this issue Apr 21, 2016

@simonmichael

This comment has been minimized.

simonmichael commented Apr 21, 2016

Great.

Would it make sense to do this automatically for any relative file path, reducing noise ?

@snoyberg

This comment has been minimized.

Owner

snoyberg commented Apr 21, 2016

I thought about that. However, that will:

  1. Break any usage outside of a cabal project (e.g., just a normal runghc call)
  2. Silently modify behavior

Possible approaches:

  1. Provide a mirror set of APIs (ugly)
  2. Same as above, and deprecate the old functions to make sure people know to upgrade
  3. Provide a special syntax in file paths (like "@foo/bar.txt"), which seems too magical
  4. Major version bump and change the semantics (which will be a silent breaking change)

I'm not feeling great about any of it TBH

@simonmichael

This comment has been minimized.

simonmichael commented Apr 21, 2016

@cies

This comment has been minimized.

cies commented May 5, 2016

I had a similar problem with htoml. Fixed by the makeRelativeToProject. Awesome!

When running stack ghci on a project that depends on htoml it was somehow building htoml's tests, which use embedDir. There error (for future reference, or easy Google'ing for those who run into this as well):

[1 of 7] Compiling Text.Toml.Types  ( /home/cies/repos/private/htoml/src/Text/Toml/Types.hs, interpreted )
[2 of 7] Compiling Text.Toml.Parser ( /home/cies/repos/private/htoml/src/Text/Toml/Parser.hs, interpreted )
[3 of 7] Compiling Text.Toml.Parser.Spec ( /home/cies/repos/private/htoml/test/Text/Toml/Parser/Spec.hs, interpreted )
[4 of 7] Compiling Text.Toml        ( /home/cies/repos/private/htoml/src/Text/Toml.hs, interpreted )
[5 of 7] Compiling BurntSushi       ( /home/cies/repos/private/htoml/test/BurntSushi.hs, interpreted )

/home/cies/repos/private/htoml/test/BurntSushi.hs:21:14:
    Exception when trying to run compile-time code:
      test/BurntSushi: getDirectoryContents: does not exist (No such file or directory)
    Code: embedDir "test/BurntSushi"
    In the splice: $(embedDir "test/BurntSushi")
Failed, modules loaded: Text.Toml, Text.Toml.Parser, Text.Toml.Types, Text.Toml.Parser.Spec.

<no location info>:
    Could not find module ‘BurntSushi’
    It is not a module in the current program, or in any known package.

Changing this:

allFiles :: [(FilePath, B.ByteString)]
allFiles = $(embedDir "test/BurntSushi")

to this:

allFiles :: [(FilePath, B.ByteString)]
allFiles = $(makeRelativeToProject "test/BurntSushi" >>= embedDir)

fixed it.

Thanks!

seagreen added a commit to seagreen/hjsonschema that referenced this issue Jun 16, 2016

Fix embedFile issue.
See here for someone with a similar problem: snoyberg/file-embed#18

hjsonschema now compiles even when referenced locally from another
project and `stack ghci` is used in that project.

Also update changelog and README.

seagreen added a commit to seagreen/hjsonschema that referenced this issue Jul 6, 2016

Fix embedFile issue.
See here for someone with a similar problem: snoyberg/file-embed#18

hjsonschema now compiles even when referenced locally from another
project and `stack ghci` is used in that project.

Also update changelog and README.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment