New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pages encoded UTF-8 with BOM not rendered correctly #1186
Comments
Have you tried including a blank line between your headings? While not strictly necessary in all situations, failing to do so is generally bad form and can result in weird edge cases. |
Actually, there is a blank line - that was a typo in writing the issue. It is the same either way. |
It seems to only be occurring on the first page (index.md) in this project, although I seem to recall it doing it on multiple pages in another project (could be wrong). |
Same Problem here :( For every document in the repository. This is the source of the file: ### Bug Template
'''
[Short description of problem here]
**Reproduction Steps:**
1. [First Step]
2. [Second Step]
3. [Other Steps...]
... Rendered HTML is <article class="md-content__inner md-typeset">
<h1>Bug template</h1>
<p>### Bug Template </p> Tested with default theme and with material in mkdocs, version 0.16.3, running on Anaconda with Python 2.7.2 on Windows 10. |
Can you post your mkdocs.yml file? Maybe this is dependent on certain extensions? I copied and pasted your example, but the syntax is weird in your example, and it doesn't look to be the full source of the image you posted. It isn't using backticks for fences, maybe that was a mistake when you posted the example? When I change the code start and end to |
I also can't reproduce the opening post's example in readthedocs. I tired a number of extension combinations and even removed all extensions, and I am not seeing that behavior. |
Hmm, sorry, I did only post the header of the document, as I thought the header declaration would suffice. This is the complete document: ### Bug Template
'''
[Short description of problem here]
**Reproduction Steps:**
1. [First Step]
2. [Second Step]
3. [Other Steps...]
**Expected behavior/content:**
[Describe expected behavior here]
**Observed behavior/content:**
[Describe observed behavior here]
**Screenshots and GIFs**
[Screenshots and GIFs which follow reproduction steps to demonstrate the problem]
**Additional information:**
* Problem started happening recently, didn't happen in an older version: [Yes/No]
* Problem can be reliably reproduced, doesn't happen randomly: [Yes/No]
* Problem happens with all files and projects, not only some files or projects: [Yes/No]
...
''' Note that I have replaced the backticks in the document with But this document was only the easiest example to use to provide the error description. The behaviour ocurrs for every document in our repository, regardless of the heading level. Every first heading is rendered as a html header ( The These are the settings currently configured: site_name: Docs
site_url: url_to_site
repo_url: repo_url
repo_name: Docs
site_description: Docs
site_author: company_name
copyright: company_name
use_directory_urls: false
theme: 'material' Note that I also used the default theme (which can't be used because of the mass of doucments, material works better for that) with the same result. I will try to generate the pages mapping automatically, perhaps this changes the result. Will keep you posted about that. |
I have tested a bit more and could identify that the problem lies with the file encoding. If the file is encoded in UTF-8 BOM (Windows 😞) then the rendering issue ocurrs. If you use proper UTF-8 there the files are generated as expected. |
Well, that's good news. At least the issue has been clearly identified. As far as I know, MkDocs doesn't allow you to specify encoding, so as long as you use proper UTF-8, you should be fine. Maybe this should be more clearly stated in the documentation if it isn't already. |
MkDocs could read files with encoding I also think it is perfectly reasonable for MkDocs to just clarify that they only accept |
That makes sense. Can't believe I didn't think of asking about encoding before.
You may have a point. I suspect this has not been done given the following in the Python docs:
First, I can't imagine anyone actually using Notepad to edit their files. Is there any other editors which actually do that by default? I've never encountered one. That said, using
So we could read using |
Yes, that is what I was suggesting, I should have emphasized read, but that was my intention with this suggestion. |
Well, I used Powershell's |
Oh course that's what Windows is doing 🤦♂️. That's such a Windows thing to do. |
Can't remember what editor i was using (usually.vscode or atom) but do occasionally make minor edits in notepad cause it opens quicker :) Not sure if autocrlf was interfering or not... |
Python simply discards the BOM with `utf-8-sig`. This way, users of Microsoft text editors can have their files properly parsered. In all other ways behaves as reading files using `utf-8` encoding. For more info see: https://docs.python.org/2/library/codecs.html#encodings-and-unicode Fixes mkdocs#1186.
Python simply discards the BOM with `utf-8-sig`. This way, users of Microsoft text editors can have their files properly parsered. In all other ways behaves as reading files using `utf-8` encoding. For more info see: https://docs.python.org/2/library/codecs.html#encodings-and-unicode Fixes mkdocs#1186.
Python simply discards the BOM with `utf-8-sig`. This way, users of Microsoft text editors can have their files properly parsered. In all other ways behaves as reading files using `utf-8` encoding. For more info see: https://docs.python.org/2/library/codecs.html#encodings-and-unicode Fixes mkdocs#1186.
Python simply discards the BOM with `utf-8-sig`. This way, users of Microsoft text editors can have their files properly parsered. In all other ways behaves as reading files using `utf-8` encoding. For more info see: https://docs.python.org/2/library/codecs.html#encodings-and-unicode Fixes mkdocs#1186.
Well, I have a fix for this in #1236, but the test fails in PyPy. I should note that only the new test fails. Presumably, everything else is working fine. PyPy simply appears to not like the file with a BOM. I can live with that, but how do I make the test pass? Should I skip the test for PyPy, or do something else? Any thoughts. |
Weird. Is it a PyPy 3.5 thing? I only have pypy 2.7 local on my mac, and I am able to write and read utf-8-sig. |
No, its the PyPy2 tests. For some reason, the output is blank. |
I may need to pull the branch and try and run it in pypy2 because it seems |
This pypy thing has me stumped. I installed pypy locally and played around a little and AFAICT, everything works except in this test. The test does the following:
In everything but PyPy, the tests works as expected. It fails before the change (MkDocs reads files with First I removed the BOM from Then I tried adding some logging statements to track values through the build process. When building, PyPy is writing out to It would seem the only failure is in step 4 above. When reading |
You beat me to it, but I was afraid that was the case. I wonder if the nosetest environment is a bit weird on pypy. I never really use pypy for anything, but it seems it has some quirks. |
Python simply discards the BOM with `utf-8-sig`. This way, users of Microsoft text editors can have their files properly parsered. In all other ways behaves as reading files using `utf-8` encoding. For more info see: https://docs.python.org/2/library/codecs.html#encodings-and-unicode Fixes mkdocs#1186.
Python simply discards the BOM with `utf-8-sig`. This way, users of Microsoft text editors can have their files properly parsered. In all other ways behaves as reading files using `utf-8` encoding. For more info see: https://docs.python.org/2/library/codecs.html#encodings-and-unicode Fixes #1186.
Hello,
I have a simple index markdown file:
And it fails to render correctly in any theme.
Cinder

ReadTheDocs

Am I doing something wrong? If I change it to a level 2 heading, the same thing happens.
I can workaround it by placing a blank line at the start of the file, but then the heading appears too low:
Thanks.
The text was updated successfully, but these errors were encountered: