New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to config.toml not detected for multiple saved edits #5205

Open
mojoa opened this Issue Sep 12, 2018 · 20 comments

Comments

Projects
None yet
5 participants
@mojoa
Copy link

mojoa commented Sep 12, 2018

Hugo fails to detect changes to config.toml when editing and saving the file multiple times. Hugo detects the first edit and issues "Change detected, rebuilding site" but does not detect further saved changes to the same file.
Fedora28 Linux and 4.14.62-70.117.amzn2.x86_64
Hugo Static Site Generator v0.48 linux/amd64 BuildDate: 2018-08-29T06:33:51Z

Steps to reproduce:
Install Hugo 4.8
run hugo newsite
In Bash window 1 run hugo server
In Bash window 2 vi config.toml and edit Title field wq
Observe hugo issue "Change detected, rebuilding site"
vi config.toml and edit Title field wq
Observe no output from hugo

Forum Discussion:
https://discourse.gohugo.io/t/live-reload-not-detecting-changes-after-first-edit/14155/4

Additional information:
Hugo does detect multiple changes to test.html in static directory but issues a different output "Static file changes detected"

@bep

This comment has been minimized.

Copy link
Member

bep commented Sep 12, 2018

@mojoa

This comment has been minimized.

Copy link

mojoa commented Oct 27, 2018

This problem still exists in in the latest DEV version. Hugo also fails to detect any error introduced into the config.toml untill hugo server is restarted.

hugo version
Hugo Static Site Generator v0.50-DEV linux/amd64 BuildDate: unknown

Also tested on AWS instance.
Watching for changes in /root/hudl/hutest/{content,data,layouts,static}
Watching for config changes in /root/hudl/hutest/config.toml
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop

Change detected, rebuilding site
2018-10-28 00:04:58.754 +0000
Total in 4 ms
^C[root@ip-172-31-21-77 hutest]# hugo server
Error: "/root/hudl/hutest/config.toml:4:1": While parsing config: (4, 1): The following key was defined twice: title
[root@ip-172-31-21-77 hutest]# uname -na
Linux ip-172-31-21-77.us-east-2.compute.internal 4.14.62-70.117.amzn2.x86_64 #1 SMP Fri Aug 10 20:14:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@ip-172-31-21-77 hutest]#

@zwbetz-gh

This comment has been minimized.

Copy link

zwbetz-gh commented Jan 3, 2019

I'm able to reproduce this with Hugo extended v0.53 on macOS High Sierra v10.13.4 and Windows v10.0.16299.

Steps:

  1. Create a new site
  2. Create layouts/index.html then add {{ .Site.Title }}
  3. Run hugo server
  4. Edit the title in config.toml. The livereload picks up the first edit. Subsequent edits require a manual page refresh in order to be picked up
@clue44

This comment has been minimized.

Copy link

clue44 commented Jan 8, 2019

Yes this is also happening on Manjaro Linux (kernel 4.19.13). Actually I remember this being an issue in Hugo in versions around 0.3 as well, and I havent used Hugo until now again, where the same issue is still present in Hugo 0.53.

When running hugo server with the -v flag, first time I make edit it gets system events:

INFO 2019/01/08 09:36:14 Received System Events:
"config.toml": RENAME "config.toml": CHMOD "config.toml": REMOVE]

(Full paths to config.toml file in the message above has been removed by me to make it shorter.)

It seems to me that the REMOVE event above is removing monitoring on the specific file being edited, because hugo only gets the events one time for each file. If I edit another file without restarting hugo, same thing happens - hugo gets the system events only for the first edit, and not future edits.

@clue44

This comment has been minimized.

Copy link

clue44 commented Jan 9, 2019

I was actually using hugo to edit other pages today and other pages seem to function normally. I can make edits and they are reloaded. Perhaps this bug only occurs when editing the config.toml file.

@anthonyfok

This comment has been minimized.

Copy link
Contributor

anthonyfok commented Jan 10, 2019

Thank you for your report. (And my apologies for my tardy follow up to this.)

See also #4701 for the original bug report and my attempted fix in #4720, commit abc5408.

Strange, I was certain that the fix #4720 used to work for me, but no longer when I recompiled it. And heck, the Hugo v0.47 release binary, the first released version to contain this fix, no longer works for me either. Strange... Anyhow, I will look into it soon.

@anthonyfok

This comment has been minimized.

Copy link
Contributor

anthonyfok commented Jan 10, 2019

Hello to all who have run into this issue! Could you please tell me the following:

  1. Your OS
  2. Your text editor
  3. The version of your text editor

Many thanks!

anthonyfok added a commit that referenced this issue Jan 10, 2019

@anthonyfok

This comment has been minimized.

Copy link
Contributor

anthonyfok commented Jan 10, 2019

See also spf13/viper#142
But note that Viper's WatchConfig() is no longer used starting Hugo v0.38. Instead of watching the web root directory, just the config.toml file is watched. However, the way a file watcher is implemented in the OS, as soon as config.toml disappears (due to a REMOVE or RENAME event), it is off the watch list, and has to be specifically added to the watch list, like what #4702 attempts to do.

On my own Debian GNU/Linux laptop running "sid", it appears that Vim's file save behaviour changed a little bit between Vim 8.0 and Vim 8.1 and Neovim. Monitoring for REMOVE event was sufficient for Vim 8.0, but in Vim 8.1 and Neovim 0.3.3, RENAME seems happens first, and the REMOVE event that comes after (?) wasn't caught.

As a quick fix, commit e3cb8e6 has been pushed to the master branch to watch for both RENAME and REMOVE events and act accordingly. That seems to solve the problem on my end, though it is not 100% robust especially when one tries to save config.toml multiple times in quick succession while Hugo is still attempting to rebuild the website. But yes, it should work for most people.

Ultimately though, I think returning to the old bahaviour of watching the Hugo web root directory (while ignoring generated directories like "public" and "resources") would be the best way forward, but until then, hope this quick fix works for all of you! Please report back and let us know if it works for you or not.

@zwbetz-gh

This comment has been minimized.

Copy link

zwbetz-gh commented Jan 11, 2019

Hi @anthonyfok -- thanks for looking into this, and giving all the background info.
Here's the info you requested:

  • OS: Windows 10
  • Text editor: Visual Studio Code v1.30.2

I just rebuilt Hugo from source and my version is:

Hugo Static Site Generator v0.54-DEV-E3CB8E6C windows/amd64 BuildDate: 2019-01-10T22:16:48-0600

I also retested on Windows 10, and I am still seeing this issue. If I have time tomorrow, I can test on macOS as well.

@bep

This comment has been minimized.

Copy link
Member

bep commented Jan 11, 2019

So, for Hugo >= 0.53 I suspect a workaround (and possibly also a solution) would be to move config.toml => config/_default/config.toml. We watch the entire config dir for changes, which should handle this issue. If you want to add the "project dir" to the watch list it must be done in a way that prevents any subfolder from being added to the watch list (i.e. no magic whitelist).

@clue44

This comment has been minimized.

Copy link

clue44 commented Jan 11, 2019

Hello to all who have run into this issue! Could you please tell me the following:

1. Your OS

2. Your text editor

3. The version of your text editor

Many thanks!

Sure but this bug will not depend on the text editor being used. I could use anything to edit the text and still reproduce the bug.

OS: Manjaro Linux 18
Text editor: Neovim 0.3.2

@clue44

This comment has been minimized.

Copy link

clue44 commented Jan 11, 2019

So, for Hugo >= 0.53 I suspect a workaround (and possibly also a solution) would be to move config.toml => config/_default/config.toml. We watch the entire config dir for changes, which should handle this issue. If you want to add the "project dir" to the watch list it must be done in a way that prevents any subfolder from being added to the watch list (i.e. no magic whitelist).

Im sure it would but then the new fixed version of Hugo would require every developer to adapt to a new location of the config.toml file, and all documentation will have to be updated to reflect the new location as well, as well as all other documentation on the web currently talking about hugo.

Personally I dont like that solution very much because of those reasons.

@bep

This comment has been minimized.

Copy link
Member

bep commented Jan 11, 2019

Sure but this bug will not depend on the text editor being used.

This particular issue is dependent on the OS/text editor combination used.

I just tested this on macOS without any issue on TextMate, VS Code, and Sublime Text 3.

@clue44

This comment has been minimized.

Copy link

clue44 commented Jan 11, 2019

Sure but this bug will not depend on the text editor being used.

This particular issue is dependent on the OS/text editor combination used.

I just tested this on macOS without any issue on TextMate, VS Code, and Sublime Text 3.

Thats really surprising to me. I figured Hugo just uses inotify to get notifications from the operating system when a file has been modified (it shouldnt matter how that modifying occurs).

But alright, let me test tomorrow with other editors as well.

@mojoa

This comment has been minimized.

Copy link

mojoa commented Jan 11, 2019

Hugo Static Site Generator v0.54-DEV linux/amd64 BuildDate: unknown
Linux fedora28.attlocal.net 5.0.0-0.rc1.git0.1.vanilla.knurd.1.fc28.x86_64

Can not replicate the problem using vi or touch using this version.

If the problem is truly inconsistent across platforms, I would think it needs a fix if possible.

As far as inotify:
Touch config.toml
./ OPEN config.toml
./ ATTRIB config.toml
./ CLOSE_WRITE,CLOSE config.toml

vi config.toml
./ OPEN config.toml
./ CREATE .config.toml.swp
./ OPEN .config.toml.swp
./ CREATE .config.toml.swx
./ OPEN .config.toml.swx
./ CLOSE_WRITE,CLOSE .config.toml.swx
./ DELETE .config.toml.swx
./ CLOSE_WRITE,CLOSE .config.toml.swp
./ DELETE .config.toml.swp
./ CREATE .config.toml.swp
./ OPEN .config.toml.swp
./ MODIFY .config.toml.swp
./ ATTRIB .config.toml.swp
./ CLOSE_NOWRITE,CLOSE config.toml
./ OPEN config.toml
./ ACCESS config.toml
./ CLOSE_NOWRITE,CLOSE config.toml
./ MODIFY .config.toml.swp
./ CREATE 4913
./ OPEN 4913
./ ATTRIB 4913
./ CLOSE_WRITE,CLOSE 4913
./ DELETE 4913
./ MOVED_FROM config.toml
./ MOVED_TO config.toml~
./ CREATE config.toml
./ OPEN config.toml
./ MODIFY config.toml
./ MODIFY config.toml
./ ATTRIB config.toml
./ CLOSE_WRITE,CLOSE config.toml
./ ATTRIB config.toml
./ MODIFY .config.toml.swp
./ DELETE config.toml~
./ CLOSE_WRITE,CLOSE .config.toml.swp
./ DELETE .config.toml.swp
./ OPEN config.toml
./ ACCESS config.toml
./ CLOSE_NOWRITE,CLOSE config.toml

@mojoa mojoa closed this Jan 11, 2019

@mojoa mojoa reopened this Jan 11, 2019

@clue44

This comment has been minimized.

Copy link

clue44 commented Jan 12, 2019

I tried editing the config.toml with Sublime Text 3.1.1 (build 3176) on my system (Manjaro Linux 18) and Hugo is rebuilding the site on every edit as it should.

But with vim it doesnt work for config.toml (works fine with all other pages though).

@anthonyfok

This comment has been minimized.

Copy link
Contributor

anthonyfok commented Jan 12, 2019

@zwbetz-gh Thank you for your report!

Here's the info you requested:

  • OS: Windows 10
  • Text editor: Visual Studio Code v1.30.2

I just rebuilt Hugo from source and my version is:

Hugo Static Site Generator v0.54-DEV-E3CB8E6C windows/amd64 BuildDate: 2019-01-10T22:16:48-0600

I also retested on Windows 10, and I am still seeing this issue. If I have time tomorrow, I can test on macOS as well.

Hmm... for some reason, I am unable to reproduce this problem on Windows 10 (64-bit) with Visual Studio Code 1.30.2. Could you please try to run hugo -v server, edit your config.toml with Visual Studio Code and save it multiple times, and post the Hugo output here? In particular, I am interested in the "Received System Events" that Hugo reports, but yes, please copy and paste Hugo's full console output here for completeness. Thank you!

@anthonyfok

This comment has been minimized.

Copy link
Contributor

anthonyfok commented Jan 12, 2019

@clue44 Thank you for your report.

I tried editing the config.toml with Sublime Text 3.1.1 (build 3176) on my system (Manjaro Linux 18) and Hugo is rebuilding the site on every edit as it should.

But with vim it doesnt work for config.toml (works fine with all other pages though).

This is with the released version of Hugo 0.53, right?

The latest development snapshot (Git master branch, 0.54-DEV with commit e3cb8e6) contains a fix that solves the problem for me with Vim on Debian GNU/Linux, so hopefully that will solve the problem for you too.

@zwbetz-gh

This comment has been minimized.

Copy link

zwbetz-gh commented Jan 12, 2019

@anthonyfok Sure thing. Here's a gist of the console output after multiple edits to config.toml with vscode.

https://gist.github.com/zwbetz-gh/b482df9cbfae9d3173af4ff56afa219f

@bep

This comment has been minimized.

Copy link
Member

bep commented Jan 12, 2019

Thats really surprising to me. I figured Hugo just uses inotify to get notifications from the operating system

inotify is a LInux "thing". We (or ´fsnotify´, the library we use) use inotify on Linux and "something else" on other OSes. But the main problem is the logic added on top of this vs the different "editor save" implementations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment