Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

./hakyll build always rebuilds everything after switching to GHC 7.8 and Hakyll 4.5.1.0 #250

Closed
TikhonJelvis opened this issue May 4, 2014 · 19 comments · Fixed by #252
Closed

Comments

@TikhonJelvis
Copy link

The caching doesn't seem to work properly any more with Hakyll 4.5.1.0 and GHC 7.8.2. Everything gets rebuilt every time, which makes using "./hakyll watch" harder.

Steps to reproduce (with my website code):

ghc -O2 hakyll.hs -o hakyll
./hakyll clean
./hakyll build # <builds everything>
./hakyll build # <builds everything again>

I tried it with older versions of my hakyll.hs file that definitely worked back on GHC 7.6.2, and the bug persisted. I also think I used to be on an older version of the Hakyll package itself, so it could be either a difference in the package or a difference in GHC.

I haven't tried with older versions of Hakyll, but I could do that if it seems useful.

Thanks for looking at this.

@blaenk
Copy link
Contributor

blaenk commented May 5, 2014

I can confirm this.

@blaenk
Copy link
Contributor

blaenk commented May 5, 2014

I'm looking into this, will update as I discover things.

For now I think I've managed to rule out the possibility of fsnotify or one of its dependencies (e.g. hinotify) being the culprit. Hakyll does only get one event for the one file that was changed, but then it goes and updates everything.

One thing I noticed is that the rulesPattern of the ruleSet returned by run is, pretty interesting. I had never observed it before so I don't know if this is normal, but I get the feeling it may point to the cause of this. Here are my patterns:

Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))) (Complement Everything)))))))))))))))))))))))))))))))))))))))) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Glob [Literal "images/",CaptureMany,Literal "",Literal ""])) (Complement (Complement (And (Complement (Glob [Literal "js/",Capture,Literal "",Literal ""])) (Complement (Complement (And (Complement (Glob [Literal "static/",CaptureMany,Literal "",Literal ""])) (Complement (Complement (And (Complement (Glob [Literal "favicon.png",Literal ""])) (Complement (Glob [Literal "CNAME",Literal ""])))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything)))))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Glob [Literal "scss/",CaptureMany,Literal ".scss",Literal ""])) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything)))))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement Everything)))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Glob [Literal "posts/",Capture,Literal "",Literal ""])) (Complement (Glob [Literal "drafts/posts/",Capture,Literal "",Literal ""]))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything)))))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Glob [Literal "notes/",Capture,Literal "",Literal ""])) (Complement (Glob [Literal "drafts/notes/",Capture,Literal "",Literal ""]))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything)))))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Glob [Literal "pages/",Capture,Literal "",Literal ""])) (Complement (Glob [Literal "drafts/pages/",Capture,Literal "",Literal ""]))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything)))))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement Everything))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement (Glob [Literal "posts/",Capture,Literal "",Literal ""])) (Complement (Glob [Literal "drafts/posts/",Capture,Literal "",Literal ""]))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything)))))))))))))))))))))) (Complement (Complement (And (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))) (Complement (Complement (And (Complement (Glob [Literal "templates/",Capture,Literal "",Literal ""])) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement Everything)))) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement (Complement (And (Complement Everything) (Complement Everything))))))))))))) (Complement Everything))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

@blaenk
Copy link
Contributor

blaenk commented May 5, 2014

I also noticed that -v says that items are out of date because they are out of date, that is, as if items depended on themselves. Perhaps this is normal, but I'd have expected that this would be more of a "because it has changed" (which it hasn't).

  [DEBUG] notes/algorithms.markdown is out-of-date because notes/algorithms.markdown is out-of-date
  [DEBUG] notes/angular.markdown is out-of-date because notes/angular.markdown is out-of-date
  [DEBUG] notes/clojure.markdown is out-of-date because notes/clojure.markdown is out-of-date
  [DEBUG] notes/cmake.markdown is out-of-date because notes/cmake.markdown is out-of-date
  [DEBUG] notes/cpp.markdown is out-of-date because notes/cpp.markdown is out-of-date
  [DEBUG] notes/erlang.markdown is out-of-date because notes/erlang.markdown is out-of-date

@jaspervdj
Copy link
Owner

@blaenk What hakyll/GHC version are you using? Looks like a real issue.

Note that something like:

[DEBUG] notes/cpp.markdown is out-of-date because notes/cpp.markdown is out-of-date

means that (Hakyll thinks) the underlying file has been modified.

@nagisa
Copy link
Contributor

nagisa commented May 5, 2014

Can also reproduce with GHC 7.8.2 with following packages inside sandbox

aeson 0.7.0.3
aeson-pretty 0.7.1
ansi-terminal 0.6.1.1
ansi-wl-pprint 0.6.7.1
asn1-encoding 0.8.1.3
asn1-parse 0.8.1
asn1-types 0.2.3
attoparsec 0.11.2.1
attoparsec-enumerator 0.3.2
base64-bytestring 1.0.0.1
base-unicode-symbols 0.2.2.4
blaze-builder 0.3.3.2
blaze-builder-enumerator 0.2.0.5
blaze-html 0.7.0.1
blaze-markup 0.6.0.0
byteable 0.1.1
bytestring-mmap 0.2.2
case-insensitive 1.2.0.0
cereal 0.4.0.1
cipher-aes 0.2.7
cipher-rc4 0.1.4
cmdargs 0.10.7
conduit 1.0.17.1
connection 0.2.0
cookie 0.4.0.1
cprng-aes 0.5.2
crypto-cipher-types 0.0.9
cryptohash 0.11.4
crypto-numbers 0.2.3
crypto-pubkey 0.2.4
crypto-pubkey-types 0.4.2.2
crypto-random 0.0.7
data-default 0.5.3
data-default-class 0.0.1
data-default-instances-base 0.0.1
data-default-instances-containers 0.0.1
data-default-instances-dlist 0.0.1
data-default-instances-old-locale 0.0.1
digest 0.0.1.2
dlist 0.7.0.1
enumerator 0.4.20
exceptions 0.5
extensible-exceptions 0.1.1.4
fsnotify 0.0.11
hakyll 4.5.1.0
hashable 1.2.1.0
hexpat 0.20.6
highlighting-kate 0.5.6.1
hinotify 0.3.6
hjsmin 0.1.4.6
hs-bibutils 5.0
hslua 0.3.10
HTTP 4000.2.12
http-client 0.3.0.2
http-client-conduit 0.2.0.1
http-client-tls 0.2.1.1
http-conduit 2.0.0.10
http-types 0.8.4
HUnit 1.2.5.2
hyphenation 0.4
language-javascript 0.5.12
lifted-base 0.2.2.1
List 0.5.1
lrucache 1.1.1.4
mime-types 0.1.0.4
mmorph 1.0.2
MonadCatchIO-transformers 0.3.1.0
monad-control 0.3.2.3
monads-tf 0.1.0.1
mtl 2.1.3.1
nats 0.1.2
network 2.4.2.2
optparse-applicative 0.8.0.1
pandoc 1.12.3.3
pandoc-citeproc 0.3.0.1
pandoc-types 1.12.3.2
parsec 3.1.5
pem 0.2.1
primitive 0.5.2.1
publicsuffixlist 0.1
random 1.0.1.1
regex-base 0.93.2
regex-pcre-builtin 0.94.4.8.8.34
regex-posix 0.95.2
regex-tdfa 1.2.0
resourcet 0.4.10.2
rfc5051 0.1.0.3
scientific 0.2.0.2
securemem 0.1.3
semigroups 0.13.0.1
snap-core 0.9.6.2
snap-server 0.9.4.2
socks 0.5.4
split 0.2.2
streaming-commons 0.1.1
syb 0.4.1
system-fileio 0.3.12
system-filepath 0.4.10
tagsoup 0.13.1
temporary 1.2.0.2
texmath 0.6.6.1
text 1.1.0.1
text-stream-decode 0.1.0.5
tls 1.2.6
transformers-base 0.4.1
unix-compat 0.4.1.1
unordered-containers 0.2.4.0
utf8-string 0.3.7
vector 0.10.9.1
void 0.6.1
x509 1.4.11
x509-store 1.4.4
x509-system 1.4.2
x509-validation 1.5.0
xml 1.3.13
yaml 0.8.8.1
zip-archive 0.2.2
zlib 0.5.4.1
zlib-bindings 0.1.1.5
zlib-enum 0.2.3

I think I could see it before GHC 7.8, so some package being at fault comes to mind.

@blaenk
Copy link
Contributor

blaenk commented May 5, 2014

@jaspervdj can't believe I forgot to mention that! GHC 7.8.2, latest
hakyll. Mainly what we mean in this thread is that the watch server
rebuilds each time, which makes things somewhat inconvenient when any one
little file is edited.

On Monday, May 5, 2014, Simonas Kazlauskas notifications@github.com wrote:

Can also reproduce with GHC 7.8.2 with following packages inside sandbox

aeson 0.7.0.3
aeson-pretty 0.7.1
ansi-terminal 0.6.1.1
ansi-wl-pprint 0.6.7.1
asn1-encoding 0.8.1.3
asn1-parse 0.8.1
asn1-types 0.2.3
attoparsec 0.11.2.1
attoparsec-enumerator 0.3.2
base64-bytestring 1.0.0.1
base-unicode-symbols 0.2.2.4
blaze-builder 0.3.3.2
blaze-builder-enumerator 0.2.0.5
blaze-html 0.7.0.1
blaze-markup 0.6.0.0
byteable 0.1.1
bytestring-mmap 0.2.2
case-insensitive 1.2.0.0
cereal 0.4.0.1
cipher-aes 0.2.7
cipher-rc4 0.1.4
cmdargs 0.10.7
conduit 1.0.17.1
connection 0.2.0
cookie 0.4.0.1
cprng-aes 0.5.2
crypto-cipher-types 0.0.9
cryptohash 0.11.4
crypto-numbers 0.2.3
crypto-pubkey 0.2.4
crypto-pubkey-types 0.4.2.2
crypto-random 0.0.7
data-default 0.5.3
data-default-class 0.0.1
data-default-instances-base 0.0.1
data-default-instances-containers 0.0.1
data-default-instances-dlist 0.0.1
data-default-instances-old-locale 0.0.1
digest 0.0.1.2
dlist 0.7.0.1
enumerator 0.4.20
exceptions 0.5
extensible-exceptions 0.1.1.4
fsnotify 0.0.11
hakyll 4.5.1.0
hashable 1.2.1.0
hexpat 0.20.6
highlighting-kate 0.5.6.1
hinotify 0.3.6
hjsmin 0.1.4.6
hs-bibutils 5.0
hslua 0.3.10
HTTP 4000.2.12
http-client 0.3.0.2
http-client-conduit 0.2.0.1
http-client-tls 0.2.1.1
http-conduit 2.0.0.10
http-types 0.8.4
HUnit 1.2.5.2
hyphenation 0.4
language-javascript 0.5.12
lifted-base 0.2.2.1
List 0.5.1
lrucache 1.1.1.4
mime-types 0.1.0.4
mmorph 1.0.2
MonadCatchIO-transformers 0.3.1.0
monad-control 0.3.2.3
monads-tf 0.1.0.1
mtl 2.1.3.1
nats 0.1.2
network 2.4.2.2
optparse-applicative 0.8.0.1
pandoc 1.12.3.3
pandoc-citeproc 0.3.0.1
pandoc-types 1.12.3.2
parsec 3.1.5
pem 0.2.1
primitive 0.5.2.1
publicsuffixlist 0.1
random 1.0.1.1
regex-base 0.93.2
regex-pcre-builtin 0.94.4.8.8.34
regex-posix 0.95.2
regex-tdfa 1.2.0
resourcet 0.4.10.2
rfc5051 0.1.0.3
scientific 0.2.0.2
securemem 0.1.3
semigroups 0.13.0.1
snap-core 0.9.6.2
snap-server 0.9.4.2
socks 0.5.4
split 0.2.2
streaming-commons 0.1.1
syb 0.4.1
system-fileio 0.3.12
system-filepath 0.4.10
tagsoup 0.13.1
temporary 1.2.0.2
texmath 0.6.6.1
text 1.1.0.1
text-stream-decode 0.1.0.5
tls 1.2.6
transformers-base 0.4.1
unix-compat 0.4.1.1
unordered-containers 0.2.4.0
utf8-string 0.3.7
vector 0.10.9.1
void 0.6.1
x509 1.4.11
x509-store 1.4.4
x509-system 1.4.2
x509-validation 1.5.0
xml 1.3.13
yaml 0.8.8.1
zip-archive 0.2.2
zlib 0.5.4.1
zlib-bindings 0.1.1.5
zlib-enum 0.2.3

I think I could see it before GHC 7.8, so some package being at fault
comes to mind.


Reply to this email directly or view it on GitHubhttps://github.com//issues/250#issuecomment-42206280
.

  • Jorge Israel Peña

@blaenk
Copy link
Contributor

blaenk commented May 5, 2014

I agree with @nagisa that it could be a dependency, though I think not hfsnotify. The reason I believe this is because I went all the way back to bc360a3 and I still observe this behavior, unless of course the problem could have been introduced before then and it's just now conflicting with GHC 7.8.2 et al.

I specifically went to that commit because it's before the global metadata series of commits, since I wondered if the recent reverts on those features were unclean and ended up causing this problem.

I should note that I've noticed that all of us in this thread seem to be using a custom Pandoc compiler, i.e. not one that is cached. Could that be causing this?

@blaenk
Copy link
Contributor

blaenk commented May 5, 2014

This is also happening on 4.4.1.0, commit 808353b. I'm starting to think it's something related to GHC 7.8.2?

@cartazio
Copy link

cartazio commented May 6, 2014

@blaenk what OS are you on? What does "ghc --info" print out?

@blaenk
Copy link
Contributor

blaenk commented May 6, 2014

@cartazio I'm on arch, with the GHC package that came out yesterday for 7.8.2. (before that was 7.6.x). Here's the output:

 [("Project name","The Glorious Glasgow Haskell Compilation System")
 ,("GCC extra via C opts"," -fwrapv")
 ,("C compiler command","/usr/bin/gcc")
 ,("C compiler flags"," -fno-stack-protector")
 ,("C compiler link flags","")
 ,("ld command","/usr/bin/ld")
 ,("ld flags","")
 ,("ld supports compact unwind","YES")
 ,("ld supports build-id","YES")
 ,("ld supports filelist","NO")
 ,("ld is GNU ld","YES")
 ,("ar command","/usr/bin/ar")
 ,("ar flags","q")
 ,("ar supports at file","YES")
 ,("touch command","touch")
 ,("dllwrap command","/bin/false")
 ,("windres command","/bin/false")
 ,("libtool command","libtool")
 ,("perl command","/usr/bin/perl")
 ,("target os","OSLinux")
 ,("target arch","ArchX86_64")
 ,("target word size","8")
 ,("target has GNU nonexec stack","True")
 ,("target has .ident directive","True")
 ,("target has subsections via symbols","False")
 ,("Unregisterised","NO")
 ,("LLVM llc command","llc")
 ,("LLVM opt command","opt")
 ,("Project version","7.8.2")
 ,("Booter version","7.8.2")
 ,("Stage","2")
 ,("Build platform","x86_64-unknown-linux")
 ,("Host platform","x86_64-unknown-linux")
 ,("Target platform","x86_64-unknown-linux")
 ,("Have interpreter","YES")
 ,("Object splitting supported","YES")
 ,("Have native code generator","YES")
 ,("Support SMP","YES")
 ,("Tables next to code","YES")
 ,("RTS ways","l debug thr thr_debug thr_l thr_p dyn debug_dyn thr_dyn thr_debug_dyn l_dyn thr_l_dyn")
 ,("Support dynamic-too","YES")
 ,("Support parallel --make","YES")
 ,("Dynamic by default","NO")
 ,("GHC Dynamic","YES")
 ,("Leading underscore","NO")
 ,("Debug on","False")
 ,("LibDir","/usr/lib/ghc-7.8.2")
 ,("Global Package DB","/usr/lib/ghc-7.8.2/package.conf.d")
 ]

@blaenk
Copy link
Contributor

blaenk commented May 6, 2014

Tired of looking into this, can't figure out any leads.

Just a heads up, however. It seems the latest version of some package, can't tell which one, but I think transformers-0.4.0.0 based on this, (since mtl is already capped at < 2.0 in hakyll, which by the way should probably be relaxed since the latest version is 2.2.0.1) depends on exceptions-0.6, which breaks temporary-1.2.0.2 (pending PR #12 that aims to fix the issue).

Suffice it to say everything seems to be a mess right now as packages scramble (hopefully) to fix their packages and their version constraints. I found that the only way to build hakyll at all was to specify an explicit constraint to cabal as follows:

cabal install --dependencies-only --constraint 'exceptions < 0.6'

That said, I still have no idea what could be causing this annoying bug.

@blaenk
Copy link
Contributor

blaenk commented May 8, 2014

This bug completely breaks my workflow, since I use hakyll nearly every day for keeping notes.

I'm willing to (and have) look into it but I'm completely at a loss as to what may be occurring. I don't have any leads to follow.

The situation so far:

  • It's curious that this bug is happening at all considering the Hakyll source hasn't really changed and the bug seems to occur with the previous versions of Hakyll that I built. So I think it must be the new GHC, its new base packages, or some other package that has since been updated.
  • The rule set patterns seem full of (And (Complement Everything) (Complement Everything)). I don't know if that's normal, since I never bothered to look at them before, but if it's not then it may be indicative of the bug.

I'm wondering if any of you have any hunches on what the issue may be, @jaspervdj perhaps?

@jaspervdj
Copy link
Owner

@blaenk I'd suggest to start digging in this method:

https://github.com/jaspervdj/hakyll/blob/master/src/Hakyll/Core/Provider/Internal.hs#L172

The way I'd go around it is just using Debug.Trace.trace to dump out everything. I suspect resourceModified is returning True where it shouldn't, but it isn't obvious to me as to why.

@blaenk
Copy link
Contributor

blaenk commented May 9, 2014

I figured it out! Will post information about the bug and a fix when I've finished.

@blaenk
Copy link
Contributor

blaenk commented May 9, 2014

Came up with the fix and submitted it as PR #251.

blaenk added a commit to blaenk/hakyll that referenced this issue May 9, 2014
It seems like System.Directory's `getModificationTime` is now providing
sub-second precision if it's available in the underlying system call,
which it seems to be for most systems.

The problem is that when the modification time is stored, the DiffTime,
the number of seconds from midnight, has its sub-second precision
shaved off, effectively becoming 0 the next time it's read.  When a
file is then checked to see if it has been modified, the file is
_always_ deemed to have been modified because this time still includes
the sub-second precision, which will almost always be later in time
than 0, unless of course it happened to be 0 to begin with.

This simply shaves the sub-second precision off of the time returned by
`getModificationTime`, and thus no longer rebuilds every single file
each time any one file is saved.

Closes jaspervdj#250
@cartazio
Copy link

cartazio commented May 9, 2014

WOAH. thats a good one

@jaspervdj
Copy link
Owner

Impressive, great debugging there!

blaenk added a commit to blaenk/hakyll that referenced this issue May 9, 2014
Some systems can get the file modification time with sub-second
granularity. However, Hakyll shaves off the sub-seconds, as defined in
the Binary instance of BinaryTime, which poses a problem because when a
file is checked to see if it was modified in `resourceModified`, it
still contains the sub-seconds. This results in a file (almost) always
being considered as having been modified.

Example:

1. First go around, modification time is 3:45.325. This time is saved
   as 3:45.000 (i.e. sub-seconds are shaved off).
2. Second go around, modification time is again read as 3:45.325 and
   compared against the stored time, 3:45.000. 3:45.325 is more recent
   than 3:45.000, so the file is considered to have been modified.

This change prevents the shaving off of sub-seconds. This will naturally
work on systems that don't support sub-second granularity, as that
'field' will simply appear as all zeros.

Closes jaspervdj#250
@blaenk
Copy link
Contributor

blaenk commented May 9, 2014

Thanks for pointing me in the right direction :)

@jaspervdj
Copy link
Owner

Released on Hackage as hakyll-4.5.2.0!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants