docpad slow to notice changes in watched files #749

Closed
trooney opened this Issue Dec 30, 2013 · 12 comments

Comments

Projects
None yet
5 participants
@trooney

trooney commented Dec 30, 2013

Docpad is slow to pick up changes to watched files. Previously a save almost immediately triggered a regeneration. Now it takes 3-9 seconds before Docpad notices the change and triggers a regeneration.

This occurs on a fresh DocPad install with only 1 document. No plugins installed.

It began around 6.55. I thought it related to the new database cache. But I've rolled back to 6.52 and the lag still remains. Have tested on my MacBook and a iMac at work. Perhaps either a change with Node, or a OS X Patch?

Docpad is a great tool, but the lag makes styling sites frustrating. Have tried adding preferredMethods watchFile and watch to docpad.coffee, but this only increases the delay.

Attached the log, but the times listed below were from an earlier run.

https://gist.github.com/trooney/8bf37b1677a8d23c6da2

docpad v6.59.6
node v0.10.24
npm 1.3.21
Macbook Pro (OS X 10.9.1, Core i7, SSD)

Change    Regenerating  Time
23:06:38  23:06:42      00:00:04
23:06:49  23:06:52      00:00:03
23:07:01  23:07:07      00:00:06
23:07:18  23:07:22      00:00:04
23:07:31  23:07:37      00:00:06
@balupton

This comment has been minimized.

Show comment
Hide comment
@balupton

balupton Dec 30, 2013

Member

This is due to a recent fix in watchr that adds a 2 second catchup delay to detect swapfile changes accurately. Previously we would fire events right away, and have a 1 second duplicate delay for catching future events, however this could not detect swapfile changes accurately so was deprecated in favour of the new functionality.

If you use an editor that uses swapfiles, you can try reducing the delay via the watchOptions.catchupDelay option, but disabling it will cause problems as swapfile changes won't be detected correctly.

If you don't use swapfiles with your editor, you can disable the catchup delay by removing any watch configuration you have already set, and adding the following to your docpad configuration file:

watchOptions:
    catchupDelay: 0

This will make it so that watch events fire immediately, rather than trying to detect duplicate events and changes to swap files.

Let me know how it goes :)

Member

balupton commented Dec 30, 2013

This is due to a recent fix in watchr that adds a 2 second catchup delay to detect swapfile changes accurately. Previously we would fire events right away, and have a 1 second duplicate delay for catching future events, however this could not detect swapfile changes accurately so was deprecated in favour of the new functionality.

If you use an editor that uses swapfiles, you can try reducing the delay via the watchOptions.catchupDelay option, but disabling it will cause problems as swapfile changes won't be detected correctly.

If you don't use swapfiles with your editor, you can disable the catchup delay by removing any watch configuration you have already set, and adding the following to your docpad configuration file:

watchOptions:
    catchupDelay: 0

This will make it so that watch events fire immediately, rather than trying to detect duplicate events and changes to swap files.

Let me know how it goes :)

@trooney

This comment has been minimized.

Show comment
Hide comment
@trooney

trooney Jan 2, 2014

Thanks for the insight! The change does narrow the time by a few seconds, so I'll close the issue.

Is there any way to make changes update near realtime? It's hard to give up near instant save-and-reloads ;)

trooney commented Jan 2, 2014

Thanks for the insight! The change does narrow the time by a few seconds, so I'll close the issue.

Is there any way to make changes update near realtime? It's hard to give up near instant save-and-reloads ;)

@trooney trooney closed this Jan 2, 2014

@balupton

This comment has been minimized.

Show comment
Hide comment
@balupton

balupton Jan 2, 2014

Member

Forgot about another delay of 100ms, try add this to your DocPad configuration file as well:

regenerateDelay: 0

Considering the recent watch changes, we may be able to get rid of that DocPad specify delay.

Member

balupton commented Jan 2, 2014

Forgot about another delay of 100ms, try add this to your DocPad configuration file as well:

regenerateDelay: 0

Considering the recent watch changes, we may be able to get rid of that DocPad specify delay.

@trooney

This comment has been minimized.

Show comment
Hide comment
@trooney

trooney Jan 2, 2014

Ah great! That's pretty much down to the old time. Thanks!!

trooney commented Jan 2, 2014

Ah great! That's pretty much down to the old time. Thanks!!

@Naatan

This comment has been minimized.

Show comment
Hide comment
@Naatan

Naatan Jan 10, 2014

This setting does not seem to be documented anywhere, makes me wonder how many other settings there are that might let us tweak the efficiency of DocPad. Should there be a bug for this? (Documenting -all- the prefs).

Naatan commented Jan 10, 2014

This setting does not seem to be documented anywhere, makes me wonder how many other settings there are that might let us tweak the efficiency of DocPad. Should there be a bug for this? (Documenting -all- the prefs).

@greduan

This comment has been minimized.

Show comment
Hide comment
@greduan

greduan Jan 10, 2014

Contributor

@Naatan It is documented actually, (http://docpad.org/docs/config, search for "regenerate"). But it isn't made clear.

I added a FAQ entry to the documentation pointing to that issue: docpad/documentation@0bc8fc8#diff-712cbca5d8e6afa5b07e4e776e6c3da3R183

Hopefully that fixes helps other users in the future. :)

Contributor

greduan commented Jan 10, 2014

@Naatan It is documented actually, (http://docpad.org/docs/config, search for "regenerate"). But it isn't made clear.

I added a FAQ entry to the documentation pointing to that issue: docpad/documentation@0bc8fc8#diff-712cbca5d8e6afa5b07e4e776e6c3da3R183

Hopefully that fixes helps other users in the future. :)

@balupton

This comment has been minimized.

Show comment
Hide comment
@balupton

balupton Jan 10, 2014

Member

watchOptions isn't documented on http://docpad.org/docs/config

+1 on resolving these things

Member

balupton commented Jan 10, 2014

watchOptions isn't documented on http://docpad.org/docs/config

+1 on resolving these things

@Naatan

This comment has been minimized.

Show comment
Hide comment
@Naatan

Naatan Jan 10, 2014

The doc only covers regenerateDelay, there's no mention of catchupDelay (which for me seemed to have a much larger impact).

I'm mostly just curious now if there are any other prefs that I might want to play with that aren't documented :)

Naatan commented Jan 10, 2014

The doc only covers regenerateDelay, there's no mention of catchupDelay (which for me seemed to have a much larger impact).

I'm mostly just curious now if there are any other prefs that I might want to play with that aren't documented :)

@greduan

This comment has been minimized.

Show comment
Hide comment
@greduan

greduan Jan 10, 2014

Contributor

@balupton Added it to the todo list. :)

Contributor

greduan commented Jan 10, 2014

@balupton Added it to the todo list. :)

@balupton

This comment has been minimized.

Show comment
Hide comment
@balupton

balupton Jan 10, 2014

Member

There may be some confusion here:

catchupDelay is a watchr configuration option, not a DocPad configuration option. Watchr configuration is set in DocPad via the watchOptions docpad configuration option.

Member

balupton commented Jan 10, 2014

There may be some confusion here:

catchupDelay is a watchr configuration option, not a DocPad configuration option. Watchr configuration is set in DocPad via the watchOptions docpad configuration option.

@Naatan

This comment has been minimized.

Show comment
Hide comment
@Naatan

Naatan Jan 10, 2014

It'd still be great to have that info in the docs :) (ie. watchOptions contains config for watchr).

Naatan commented Jan 10, 2014

It'd still be great to have that info in the docs :) (ie. watchOptions contains config for watchr).

@ghost ghost assigned greduan Jan 10, 2014

@greduan greduan referenced this issue in docpad/documentation Jan 10, 2014

Closed

Document watchOptions #94

balupton added a commit to docpad/documentation that referenced this issue Jan 20, 2014

@Zearin

This comment has been minimized.

Show comment
Hide comment
@Zearin

Zearin Apr 19, 2015

Member

Was getting the “regenerate on first change, then nothing” problem myself. Found this issue.

For @trooney and anyone else interested in speeding up performance:

The “official” template engine is usually docpad-plugin-eco. However, it has a fork named docpad-plugin-ect, which is focused on performance. It uses the same syntax as ECO.

If you switch the plugins, remember to rename all your *.eco to *.ect. :)

Finally, here are some benchmarks that compare the performance of ECT to ECO.

Note: (Core) ECT includes some features beyond that of (core) ECO. However, these aren’t available in docpad-plugin-ect. They are mentioned in that link, so I just wanted to be clear. This is a difference between the core template engines; there are no feature differences between docpad-plugin-eco and docpad-plugin-ect (only performance).

Member

Zearin commented Apr 19, 2015

Was getting the “regenerate on first change, then nothing” problem myself. Found this issue.

For @trooney and anyone else interested in speeding up performance:

The “official” template engine is usually docpad-plugin-eco. However, it has a fork named docpad-plugin-ect, which is focused on performance. It uses the same syntax as ECO.

If you switch the plugins, remember to rename all your *.eco to *.ect. :)

Finally, here are some benchmarks that compare the performance of ECT to ECO.

Note: (Core) ECT includes some features beyond that of (core) ECO. However, these aren’t available in docpad-plugin-ect. They are mentioned in that link, so I just wanted to be clear. This is a difference between the core template engines; there are no feature differences between docpad-plugin-eco and docpad-plugin-ect (only performance).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment