Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc anchor doesn't work for strings which need to encode in URL #458

Closed
2 tasks done
stzn opened this issue Jan 10, 2023 · 1 comment
Closed
2 tasks done

doc anchor doesn't work for strings which need to encode in URL #458

stzn opened this issue Jan 10, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@stzn
Copy link
Contributor

stzn commented Jan 10, 2023

Description

I use Japanese to headings like "## テスト" in a markdown, and I'd like to use it for an anchor like "doc:SomeMarkdown:テスト". But it doesn't work.

If I change "テスト" to "test" ( "doc:SomeMarkdown:test"), it works well.

Related forums thread: https://forums.swift.org/t/can-docc-auto-generate-anchors-be-used-in-non-ascii-strings/62295

Checklist

  • If possible, I've reproduced the issue using the main branch of this package.
  • This issue hasn't been addressed in an existing GitHub issue.

Expected Behavior

Strings which need to encode in URL for a heading can be used as an anchor.

Actual behavior

URL does encoding of the fragment identifier, but there is no encoding done in the id attribute of the HTML for the heading.

For example,

06e755868c71be62aaf1230663260b9f9d20e656

4c73610f544c4a7d82016f43bd950d7243529ab9

I confirmed it in swift-book repository.

Steps To Reproduce

  1. Use strings which need to encode in URL for a heading in a markdown
  2. Use it as an anchor.

Swift-DocC Version Information

main branch

Swift Compiler Version Information

swift-driver version: 1.62.15 Apple Swift version 5.7.1 (swiftlang-5.7.1.135.3 clang-1400.0.29.51)
Target: x86_64-apple-macosx13.0
@stzn stzn added the bug Something isn't working label Jan 10, 2023
@stzn
Copy link
Contributor Author

stzn commented Jan 15, 2023

As far as I confirmed, the difference between cmark-gfm and swift-docc probably causes this issue.

When creating the value of the href in cmark-gfm, it escapes characters except "-_.+!*'(),%#@?=;:/,+&$~ alphanum". So "テスト" was escaped.
https://github.com/github/cmark-gfm/blob/6a6e335709ef68cf2c616eeaf61b09ed4c654669/src/houdini_href_e.c#L46

But, according to RFC3986, Japanese are allowed, so URLComponents(string:) can be instantiated and "テスト"  is outputted as it is. That's why a valid ValidatedURL could be returned here.
https://github.com/apple/swift-docc/blob/f11ea837fb1db953b98707c01ab671ccc0d38dbf/Sources/SwiftDocC/Utility/ValidatedURL.swift#L60

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants