-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Improve org mode support #5545
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve org mode support #5545
Conversation
parser/pageparser/pagelexer_intro.go
Outdated
// contrary to other document and front matter formats, part of the main section. | ||
// l.emit advances the offset (start) of the lexer (and thus excludes the front matter | ||
// from the main section) so we have to reset it. | ||
frontMatterStart := l.start |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't look right. I don't understand the comment, but even if I did: This will produce more content than it consumes, and I'm fairly confident that this can be solved without this workaround.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try to explain my goal, maybe you can point me into the right direction:
I want hugo to send the complete file contents to go-org - not just what hugo calls the MainSection (the part after the front matter). This change means to, in a way, use the front matter section twice: (1) for the hugo front matter and (2) as part of the actual content (main section) that is rendered / send to go-org
.
The reason for this is that the front matter possibly contains information that is important for the rendering as #+FOO: bar
is the settings format of the org-mode parser.
An example:
#+TITLE: Foo
#+TODO: TODO DONE CUSTOM
content [...]
Hugo currently only sends
content [...]
to the renderer. However, the #+TODO: TODO DONE CUSTOM
is actually configuration for the Org mode renderer - go-org
would use it to update the list of keywords that are recognized as TODO keywords in headlines - see the headline with CUSTOM todo status example.
So I guess the problem this change is trying to solve is that the front matter of an org file actually serves two purposes: (1) configure hugo (2) configure the Org mode parser.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could store away the start/end positions for front matter here:
https://github.com/gohugoio/hugo/blob/master/hugolib/page_content.go#L92
We currently don't use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I guess the problem this change is trying to solve is that the front matter of an org file actually serves two purposes: (1) configure hugo (2) configure the Org mode parser.
Why not send the unmarshaled frontmatter?
https://github.com/gohugoio/hugo/blob/master/hugolib/page_content.go#L94
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! this sounds like it will get more involved than a 2 line change...
As the benefit of this is pretty small for now (there's not much to be configured in go-org for now) I suggest we split this into a separate PR and just focus on org mode support itself for now.
Is that ok with you?
Why not send the unmarshaled frontmatter?
Good idea - but this requires some more work on the go-org side first - as already discussed in the forum Org mode itself does not support anything but plain string values [1] - so right now we have a problem mapping hugo front matter map[string]interface{}
to the map[string]string
org mode understands. I am not sure how I want to go ahead with this yet and would like to think about it some more.
[1] The approach suggested by kaushalmodi is custom to ox-hugo
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As the benefit of this is pretty small for now
Agree. And Hugo's API in this area is also rather clumsy. I have been planning on redoing that for a while ... But time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also note that If you need some "configuration for the rendererd", we should add that as configuration in Hugo which you could receive in some constructor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also note that If you need some "configuration for the rendererd", we should add that as configuration in Hugo which you could receive in some constructor.
Maybe I shouldn't have called it configuration for the renderer. It's not configuration in the sense that it should be passed to the constructor. Rather, we currently remove part of the org document because we conflate hugo configuration (front matter) and org mode configuration (buffer settings) by using the same syntax (#+KEY: VALUE
), location (top of the file) and namespace for both of them.
So by removing the front matter from the content that is send to the renderer we actually remove a part of the document.
Coming back to the point before
Why not send the unmarshaled frontmatter?
I like the idea because it sounds like a good way to provide go-org with the full document without polluting the hugo code base with org specific stuff - having the front matter available sounds like it could be useful for other things as well. But, ignoring hugo, just sending the content + the raw front matter together to go-org would be the best thing to do.
I hope that makes sense...
Updated with support for |
I'm traveling at the moment. I'll test out this branch once I get back home (Jan 3). |
I've started exploring hugo as a platform for my blog. As Emacs user, the native support for org mode was definitely +1 when selecting the tool for the job. I've already hit limitations of Not that it would matter much, but I've checkout this branch and rendering of org-mode work just fine for me: looking forward for hugo having an active maintenance of the org-mode backend again. |
Good point! I didn't know about It doesn't seem to result in any errors Also Org mode expects at least some keywords to contain square brackets. see
|
Thanks for checking that.
And I did not know that. Good to know :) About testing go-orgAs I wasn't a Goorgeous user, I did not have any handy posts in Org format. But I know that @zamansky uses it and blogs often. So I built
Looks like you just need to support Awesome job! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good except for that one minor feature request.
Awesome, thanks for testing it out :)! Regarding the warnings:
To summarize, as far as i can tell, neither is a bug in [1] I don't know how the blog is rendered as it is at https://cestlaz-nikola.github.io, but using hugo with goorgeous actually swallows everything after the opening tag of the source block in question. If you want to try it out yourself the path to the post is [2] Hugo treats lines starting with |
I would have thought that |
I'm not sure if I understand you correctly, could you expand on that? Maybe there's a misunderstanding: What you quote was meant to be about how The warning in question is generated by a file starting with
Hugo thinks the first line is front matter (as it starts with
As there's no PS: This did point to a bug in my integration of go-org into hugo - while i did set the logger to jww for the content rendering, i did not do so for the front matter parsing. Due to this there's also a log message from go-org not prefixed by |
Yes, I quoted correctly. Sorry for not being clear. I meant to ask what |
Thx, got it now :). Hugo treats the It's just about where in the file the line is, nothing specific about the The fix for the above case is simply
|
Understood. Thanks. That means that Update: Answering myself to the above question: Yes, based on your edit of your comment above. This is a separate issue then for hugo.. it should consider |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me.
👍
👍 Thanks for taking the time to review this :) PS: The last force push just fixes the front matter logging & updates |
You are officially the most qualified person knowing both Go and Org. Do you think you can tackle fixing that in |
I have it on my todo list (along with #5436) :). |
@bep Do you have further comments on this PR? |
@bep Another tiny nudge to review this. |
Sadly, goorgeous has not been updated in over a year and still has a lot of open issues (e.g. no support for nested lists). go-org fixes most of those issues and supports a larger subset of Org mode syntax.
Hugo requires some front matter values to be arrays (e.g. for taxonomies). Org mode front matter syntax (`#+KEY: VALUE`) does however not support anything but string values normally - which is why goorgeous hardcoded the values for the keys tags, categories & aliases to be parsed as string arrays. This causes problems with custom taxonomies. A simple thing we can do instead is make keywords ending in '[]' be parsed as string arrays.
@bep is there anything specific blocking this? i can totally understand if this just doesn't have priority, just stumbled across this again by chance :) |
@niklasfasching really sorry about this. I was reminded about this when I got some version issues with that other library. |
No worries, thanks for all your work on hugo :) |
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
As the title says, this PR improves the Org mode support of hugo. For a basic overview of the changes, check out this page - it showcases the html output of
goorgeous
(left, currently included with hugo) andgo-org
(right, this PR) for the same input. The css is adapted togo-org
, so open up dev tools for a more fair comparison.For an overview of the open issues of
goorgeous
that are fixed ingo-org
take a look hereIf you want to try out any Org mode snippets, you can do so in your browser here.
Also take a look at the discussion for more information.
I included a second commit in this PR that adds the Org mode front matter to the main content as Org mode itself also uses the front matter for configuration - the impact of this is much smaller (as go-org only uses the
TODO
front matter variable right now - so we could move that to a separate PR.