Org export issue: unable to resolve link "configuring-spacemacs"

[ztlevi]

I got an issue when I try to export org to other formats.

Does anybody know this issue?
Thanks.

[d12frosted]

I just tried to export to html and everything worked as expected. Could you please provide more clear reproduction steps? Also, having output of SPC h d s would be helpful.

[ztlevi]

This is my config: https://github.com/ztlevi/Spacemacs-config

System Info 💻

• OS: darwin
• Emacs: 25.1.1
• Spacemacs: 0.200.5
• Spacemacs branch: develop (rev. 703733b)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: ivy
• Layers:

(php ivy better-defaults github ranger colors prodigy search-engine graphviz
     (syntax-checking :variables syntax-checking-enable-by-default nil syntax-checking-enable-tooltips nil)
     (spell-checking :variables spell-checking-enable-by-default nil)
     (vinegar :variables vinegar-reuse-dired-buffer t)
     (spacemacs-layouts :variables layouts-enable-autosave nil layouts-autosave-delay 300)
     (git :variables git-magit-status-fullscreen t magit-push-always-verify nil magit-save-repository-buffers 'dontask magit-revert-buffers 'silent magit-refs-show-commit-count 'all magit-revision-show-gravatars nil)
     (ibuffer :variables ibuffer-group-buffers-by 'projects)
     (auto-completion :variables auto-completion-enable-sort-by-usage t auto-completion-enable-snippets-in-popup t :disabled-for org markdown)
     (osx :variables osx-dictionary-dictionary-choice "Simplified Chinese - English")
     restclient
     (gtags :disabled-for clojure emacs-lisp javascript latex python shell-scripts)
     (shell :variables shell-default-shell 'eshell)
     docker deft markdown org shaders yaml react
     (python :variables python-test-runner
             '(nose pytest))
     (ruby :variables ruby-version-manager 'chruby)
     ruby-on-rails lua html javascript
     (typescript :variables typescript-fmt-on-save nil typescript-fmt-tool 'typescript-formatter)
     emacs-lisp
     (clojure :variables clojure-enable-fancify-symbols t)
     racket
     (c-c++ :variables c-c++-default-mode-for-headers 'c++-mode)
     zilongshanren
     (chinese :packages youdao-dictionary fcitx :variables chinese-enable-fcitx nil chinese-enable-youdao-dict t)
     dash)

• System configuration features: JPEG RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES
  Thanks.

[nikolaiam]

I can reproduce this. The problem is related to the links in Org files referencing other parts of the same documents, such as the link on line 9 in CONTRIBUTING.org. Once these links are removed the file exports perfectly.

Reproduction steps:

• Do a clean Spacemacs install, enable org layer
• Open CONTRIBUTING.org
• Press , e h H
• Result: user-error: Unable to resolve link: "asking-for-help"

System Info 💻

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.200.7
• Spacemacs branch: master (rev. c2774bc)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(helm emacs-lisp org)

[smile13241324]

Since org 9 internal links must be declared either as #customID or textTargetOrFullTextSearch. Relative path links must be either file:relative/path or ./relative/path. Many links are using the #customID form to trigger a full text search which is not defined as to the org documentation. In addition relative file paths are declared as relative/path which is also not defined. If this is done the html export can fail. Therefore the links must be corrected for the affected files.

See http://orgmode.org/manual/External-links.html#External-links and http://orgmode.org/manual/Internal-links.html#Internal-links

[smile13241324]

I will have a look at the affected documentation files and correct the links if necessary.

[ztlevi]

All right. Thanks.
I reinstall spacemacs and the problem solved.

[deb0ch]

Just updated to latest develop and updated packages, and I am still experiencing a similar bug.

More importantly, this bug happens with auto-generated table of contents, so even if it's related to something changed in org-mode 9 API this should not happen and is definitely a bug.

Repro steps:

• write an org file with some headings
• insert an auto-generated table of contents
• try to export to html using org-dispatch
• user-error: Unable to resolve link: "firstheading"

[smile13241324]

This is becoming even stranger, when I do as you say the links generated in my TOC are invalid when I try to follow them in org-mode. However when I export the document as html it works as expected.

I agree with you this really looks now like a bug in org mode.

Anyway in the meantime I have went through the spacemacs documentation files and corrected some relative links as well as adding Custom_IDs for some headlines which org failed to find during export, so that on my branch I am able to export all the org files in the project root as well as the doc folder again.

Maybe we can use this to kill the symptoms of the issue until it is solved.

[syl20bnr]

Does it still happen on current develop ?

[deb0ch]

Just rebased current develop and updated packages, still happens (I mean, the bug with org-mode).

When I try to export to html an org file that I wrote myself (with , e e h o in org-mode), it fails saying that it was unable to resolve the first link of my table of contents.

[deb0ch]

Although it doesn't happen anymore in CONTRIBUTING.org 😸

[nikolaiam]

At my end the TOC links export normally. @deb0ch, might there be something in particular in your .org file that breaks the export? In particular, how do you declare the TOC? In my case, the line has to be :TOC:noexport: to work.

[deb0ch]

I'm just using :TOC:. Indeed, I just tried with :TOD:noexport: and it works 😮

This is so fucked up, since :noexport: is documented as being for excluding the concerned item from any export (unconditionally sic), and my table still got exported...

I don't understand that thing. Is it a bug ? Or is it intentional ? 😕

[nikolaiam]

I dunno, just found out about this when trying to reproduce your behaviour 😄. But I guess it might be a bug in toc-org.

[deb0ch]

@nikolaiam Reporting, thanks for the link 😸

EDIT: reported: snosov1/toc-org#35

[smile13241324]

Just wanted to start the discussion again what to do next after snosov1 answered to our issue.

The bottom line seems to be that the org html export is not compatible with the link style generated by toc-org.

I have tried the proposed solution of using the org hrefify function by using the alternative toc tag but this only works if we generate a toc for the two headline levels only.

The noexport tag is actually fixing the issue because this triggers toc-org to remove the toc for the html export which then creates its own toc.

So should we now try to raise a bug for the org html export or provide our own hrefify function or simply ensure that all tocs are also having the noexport flag?

Independent of the outcome of this discussion I think we should write a test to ensure that all org files can be generated to html and that all links are valid.

[syl20bnr]

@smile13241324
Not sure if it is relevant for this discussion, I'm not 100% sure to understand the documentation generation process in Spacemacs, maybe @JAremko can provide more info than what follows.

In order to support everywhere links generated by both GitHub org renderer and org html renderer we first chose to use the GitHub format in org files written for Spacemacs so links work on GitHub, then to make them to work on spacemacs.org we unhrefify the links before exporting them to html files with org, this may be the same kind of strategy you propose. You can find the functions there: https://github.com/syl20bnr/spacemacs/blob/develop/core/tools/spacefmt/toc-org.el#L156-L252

Hope this helps :-)

EDIT: rewrote some sentences to make them more understandable.

[JAremko]

@syl20bnr We attach org-mode CUSTOM_ID property annotation to each headline. It makes org-mode export use CUSTOM_ID: value in URL's Then we export and mark buffer as unmodified (to avoid saving properties)

https://github.com/syl20bnr/spacemacs/blob/master/core/core-documentation.el#L111

[smile13241324]

Finished the test for the org html export. It ensures that all documentation org files can be converted to html and that all contained external links are valid.

@JAremko could you double check that org-html-export-as-html is the right function to export to html? Until now I have thought that we would use the standard org-html-export. However now after I have read your comment and have seen your spacemacs//org-heading-annotate-custom-id I am no longer so sure. For example with the export I have used, one still need to add the custom-id by hand.

Maybe if you could explain the differences between the spacemacs specific and the standard org html export a bit more I could fine tailor the test to your needs.

See merge request: #8431

[JAremko]

@smile13241324 You'll need advices from spacemacs/publish-doc

Maybe we can move them from this function and use globally for org files marked as Spacemacs documentation.

[smile13241324]

@JAremko This may not be necessary, I will simply execute spacemacs/publish-doc and then direct the html check code against all html files in the export directory.

[smile13241324]

Finished the test, related PR: #8431

For travisCI I have deactivated the external link check for now as this seems to exceed a bandwith limit which will cause the job to stop following links. Needs to be executed locally.