New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework and consolidate the page parser #5324

Closed
bep opened this Issue Oct 16, 2018 · 1 comment

Comments

Projects
None yet
1 participant
@bep
Member

bep commented Oct 16, 2018

Working on #5284 I hit some hurdles when I wanted to show proper file/line context for errors in content files. The main problem is that the current parser splits front matter and the rest. There are currently some line number calculations in there, but it seems to be impossible to get exactly right in every situation.

The current "front matter" parser is also a little hard to read, and while considered stable and working, It is easy to read from the Git history that there have been corner cases that is simpler to avoid with a proper lexer.

So I suggest that:

  • Move the shortcode parser to the parser package
  • Extend that parser to handle front matter tokens
  • Return that AST to the client with the full and original content byte slice (possibly with some shortcut methods to get the front matter etc.)

There is also an obvious potential performance benefit in the "do it once" approach above.

@bep bep added the Enhancement label Oct 16, 2018

@bep bep added this to the v0.50 milestone Oct 16, 2018

@bep bep self-assigned this Oct 16, 2018

@bep bep changed the title from Rework the page parser to Rework and consolidate the page parser Oct 16, 2018

bep added a commit to bep/hugo that referenced this issue Oct 16, 2018

commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes gohugoio#5284
Fixes gohugoio#5290
See gohugoio#5325
See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 16, 2018

commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes gohugoio#5284
Fixes gohugoio#5290
See gohugoio#5325
See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 16, 2018

commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes gohugoio#5284
Fixes gohugoio#5290
See gohugoio#5325
See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 16, 2018

commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes gohugoio#5284
Fixes gohugoio#5290
See gohugoio#5325
See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 16, 2018

commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes gohugoio#5284
Fixes gohugoio#5290
See gohugoio#5325
See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 16, 2018

commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes gohugoio#5284
Fixes gohugoio#5290
See gohugoio#5325
See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 17, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 18, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

hugolib: Redo the summary delimiter logic
Now that we have a proper page parse tree, this can be greatly simplified.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

Convert the rest to new page parser code paths
And remove some now unused code.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

hugolib: Redo the summary delimiter logic
Now that we have a proper page parse tree, this can be greatly simplified.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

Convert the rest to new page parser code paths
And remove some now unused code.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 22, 2018

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

herrors: Improve handling of JSON errors
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.

This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

herrors: Improve handling of JSON errors
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.

This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

herrors: Improve handling of JSON errors
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.

This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

herrors: Improve handling of JSON errors
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.

This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.

See gohugoio#5324

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

bep added a commit to bep/hugo that referenced this issue Oct 23, 2018

@bep

This comment has been minimized.

Member

bep commented Oct 24, 2018

This is now done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment