New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qAddDependentFile doesn't cause re-compilation without extra-source-files #4360

Closed
pbrisbin opened this Issue Oct 16, 2018 · 13 comments

Comments

Projects
None yet
5 participants
@pbrisbin
Copy link

pbrisbin commented Oct 16, 2018

Steps to reproduce

  1. Set up a repro project with stack new --bare repro

  2. Add a file templates/example.hamlet

    <h1>Hi
  3. Update Lib.hs to

    {-# LANGUAGE TemplateHaskell #-}
    
    module Lib
       ( someFunc
       ) where
    
    import Text.Blaze.Html.Renderer.Pretty
    import Text.Hamlet
    
    someFunc :: IO ()
    someFunc = putStrLn $ renderHtml $(shamletFile "templates/example.hamlet")

    (Add the shakespeareand blaze-html dependencies as necessary.)

  4. Start up stack build --file-watch

  5. Modify the contents of templates/example.hamlet

Expected

Modifying templates/example.hamlet should trigger re-compilation.

Actual

It does not. However, if I add

extra-source-files:
  - templates/*

To package.yaml, it works as expected. This should not be necessary as this packaging field is only documented as related to including files in sdist, not as a requirement for qAddDependentFile to work.

Stack version

$ stack --version
Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2

Method of installation

https://aur.archlinux.org/packages/stack-static/

Notes

It's definitely an assumption on my part that this boils down to qAddDependentFile. So I apologize if that's not actually the case. Another way to phrase it could be, external files referenced in TemplateHaskell don't cause re-compilation....

I've reproduced this bug with pretty much every file-reading TH function in a standard Yesod app (parseRoutesFile, widgetFile, etc).

I wouldn't have guessed this was Stack's responsibility either, but cocreature in #haskell encouraged me to report this bug here.

@dbaynard

This comment has been minimized.

Copy link
Contributor

dbaynard commented Oct 16, 2018

Thanks for the report. You may wish to try the new release of stack but I suspect this hasn't changed Edit: see below.

this packaging field is only documented as related to including files in sdist

This is the Cabal documentation, right?

@snoyberg how should we proceed?

@snoyberg

This comment has been minimized.

Copy link
Contributor

snoyberg commented Oct 17, 2018

I'm on my phone and can't search, but I'm fairly certain this is a bug that's been fixed on master. There was a change in the format of output from GHC.

@pbrisbin

This comment has been minimized.

Copy link

pbrisbin commented Oct 19, 2018

@dbaynard

This is the Cabal documentation, right?

Yeah, I was referring to this: https://www.haskell.org/cabal/users-guide/developing-packages.html#pkg-field-extra-source-files

@dbaynard

This comment has been minimized.

Copy link
Contributor

dbaynard commented Oct 21, 2018

@pbrisbin Do check the new version, when you get a chance — that should fix it.

@pbrisbin

This comment has been minimized.

Copy link

pbrisbin commented Oct 22, 2018

Unfortunately, this still reproduces for me with 1.9.

(In the upper-right you can see me modifying the template, but no re-compilation happens in the lower-right -- modulo that first initial compilation.)

@dbaynard

This comment has been minimized.

Copy link
Contributor

dbaynard commented Oct 22, 2018

Ok, I can reproduce

@snoyberg

This comment has been minimized.

Copy link
Contributor

snoyberg commented Oct 22, 2018

The change that should fix this is only on the master branch, not on 1.9.

@ygale

This comment has been minimized.

Copy link

ygale commented Oct 23, 2018

It's this commit: aae15e3

I wishfully submitted #4369, hoping that this small but important fix might be patched to 1.9.3.

@ygale

This comment has been minimized.

Copy link

ygale commented Oct 24, 2018

Unfortunately, it appears that fix does not solve this issue.

In GHC 8.2 and earlier, the .dump-hi file contains addDependentFile lines for the TH files. That is no longer the case in GHC 8.4.

@ygale

This comment has been minimized.

Copy link

ygale commented Oct 25, 2018

The workaround suggested by @pbrisbin to add the templates to extra-source-files does not work for me. Stack does then notice that the template file has changed, but still does not rebuild the Haskell module that uses the template.

Stack has lost the association between template file and Haskell module. This is because starting in GHC 8.4, GHC does not seem to report template files in the .dump-hi file, which is where stack looks for it.

If you add the template file to extra-source-files and also pass the -fforce-recomp GHC option, then stack will notice the change and rebuild all modules in the entire project. I can't find any way to get stack to rebuild just the affected module as in GHC 8.2 and below.

@ygale

This comment has been minimized.

Copy link

ygale commented Oct 25, 2018

Hmm shakespeare explicitly reports its files to GHC as dependent files. This is starting to look like a GHC bug. I will investigate further.

@tolysz

This comment has been minimized.

Copy link
Collaborator

tolysz commented Oct 25, 2018

@ygale only the version shakespeare-2.0.19+ will report it.

@ygale

This comment has been minimized.

Copy link

ygale commented Oct 25, 2018

@tolysz Ahh yes that was it. Thanks!!

In summary: To resolve this issue you need:

  • stack >= 1.9.3
  • shakespeare >= 2.0.19

As of this writing stack 1.9.3 is not yet released. For now, you can build stack 1.9.2 from the stable branch on github and it will work.

If you use Stackage LTS, upgrade to at least 12.14, currently the latest, to get the right version of shakespeare. Or specify shakespeare-2.0.19 in the extra-deps section of your stack.yaml.

A million thanks to the stack team for getting this fixed!

@dbaynard dbaynard closed this Nov 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment