Cross-reference problems with Pandoc 2.8 #832

Closed
opened this issue Nov 27, 2019 · 13 comments
Closed

Cross-reference problems with Pandoc 2.8#832

opened this issue Nov 27, 2019 · 13 comments
Labels

1. Cross-references are not correctly generated

For simple Rmd-document shown below:

---
title: Cross-reference
site: bookdown::bookdown_site
---

# Part 1 {#p1}

## Chapter 1 {#p1ch1}

Some text.

## Chapter 2 {#p1ch2}

Here we need to refer to chapter 1 as  \@ref(p1ch1) : see ch. \@ref(p1ch1).

I got broken cross-reference with

Rscript -e "bookdown::render_book('cross-reference.Rmd', 'bookdown::html_document2')"

so it looks as follows:

(please note "?? in output above). All often-used writers such as HTML, EPUB, ODT and DOCX are affected. But the PDF is not affected.

Note: this is repost of jgm/pandoc#5941 and the Pandoc author said that this is not their issue. The problem is caused only by pandoc 2.8. As all other related components remain the same.

2. Figure, table, example and equation numbering changed from two-level to single-level in HTML output

For simple Rmd-document shown below:

---
title: Item numbering
site: bookdown::bookdown_site
---

# Figure

![(\#fig:md-logo) Markdown logo](Markdown-mark.svg.png)

# Table

Table: (\#tab:results) Demo table

|------------|------------|
| r1, c1     | r1, c2     |
| r2, c1     | r2, c2     |

# Equation

$$f\left(k\right) =\binom{n}{k} p^k\left(1-p\right)^{n k} (\#eq:binom)$$

# Example

{example, simple-code}
Simple code


int main() {
return 0;
}


I got single-level numbering of objects which is not expected for me:

Rscript -e "bookdown::render_book('numbering.Rmd', 'bookdown::html_document2')"

ends with rendering below:

(with pandoc 2.7.3 numbering had two-levels)

Note: this is a repost of jgm/pandoc#5942. The problem is caused only by pandoc 2.8. As all other related components remain the same. The EPUB writer is affected too.

My version info:

> xfun::session_info('bookdown')
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS, RStudio 1.2.5019

Locale:
LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=ru_RU.UTF-8
LC_COLLATE=en_US.UTF-8     LC_MONETARY=ru_RU.UTF-8    LC_MESSAGES=en_US.UTF-8
LC_TELEPHONE=C             LC_MEASUREMENT=ru_RU.UTF-8 LC_IDENTIFICATION=C

Package version:
base64enc_0.1.3 bookdown_0.16   digest_0.6.23   evaluate_0.14   glue_1.3.1      graphics_3.4.4
grDevices_3.4.4 highr_0.8       htmltools_0.4.0 jsonlite_1.6    knitr_1.26.1    magrittr_1.5
markdown_1.1    methods_3.4.4   mime_0.7        Rcpp_1.0.3      rlang_0.4.2     rmarkdown_1.18
stats_3.4.4     stringi_1.4.3   stringr_1.4.0   tinytex_0.17    tools_3.4.4     utils_3.4.4
xfun_0.11       yaml_2.2.0
>

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
• If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
• If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('bookdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/bookdown').
• If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

yihui commented Nov 27, 2019

 Thanks for the report, but I probably won't have time to investigate this in the near future. Any particular reason why you need to use the very latest version of Pandoc instead of the version bundled in RStudio?
changed the title problem with pandoc 2.8: cross-references inside document are broken Cross-reference problems with Pandoc 2.8 Nov 27, 2019
mentioned this issue Nov 27, 2019

RLesur commented Nov 28, 2019 • edited

For bookdown::html_document2, I think problem 1 comes from new attributes in headings.
With Pandoc 2.8 and 2.8.0.1:

pandoc -N --section-divs -t html4 <<< "# Hello"

gives

<div id="hello" class="section level1">
</div>

I guess this regex should be modified

Line 371 in e5b2d7f

 reg_num = '^()([.A-Z0-9]+)(.+).*\$'

BTW, the goal of Pandoc 2.8 was to write:

<div id="hello" class="section level1">
</div>

N0rbert commented Nov 28, 2019 • edited

 Any particular reason why you need to use the very latest version of Pandoc instead of the version bundled in RStudio? I use GitLab CI/CD pipeline with latest pandoc to have fresh functionality. So it is head-less mode without RStudio. I'll stay with pandoc 2.7.3 to prevent unpredictable behaviour. I think problem 1 comes from new attributes in headings. Thanks, friends. I appreciate your efforts :) I'll be very happy if we have stable bookdown<->pandoc experience.

yihui commented Nov 28, 2019

 @N0rbert I don't know about GitLab's CI, but on Travis, you can easily specify the Pandoc version: https://docs.travis-ci.com/user/languages/r/ @RLesur Thanks for the tip! I think I should be able to fix the first problem soon.

N0rbert commented Nov 28, 2019 • edited

 @yihui Thank you for info about Travis! How can I help with debugging the second problem?

RLesur commented Nov 28, 2019 • edited

 I was wrong about data-number, see jgm/pandoc#5944 (comment). If Pandoc is fixed and removes the extra number attribute, there will be no need for a fix in bookdown for the first problem.

RLesur commented Nov 28, 2019

 @N0rbert If you are interested in, I also used GitLab CI to debug the first problem. My repo is here: https://gitlab.com/RLesur/bookdown-issue832.

N0rbert commented Nov 28, 2019

 @RLesur I'm interested, but I currently I can't get all the debugging information on my system because of lack of knowledge. In YAML currently I'm using output: bookdown::html_document2: pandoc_args: ['--trace', '--verbose', '--log=pandoc.log'] clean: false keep_md: true and build with Rscript -e "bookdown::render_book('cross-reference.Rmd', 'bookdown::html_document2', clean=FALSE)" but it does not highlight the problem. @yihui Even after looking thorough on I can't understand which application converts \@ref(...) to html-links. How can we debug this? How can I run all the steps manually one-after-one? How should we determine which of bookdown and/or pandoc is really causes the problem? What I can see from user point-of-view - installing pandoc 2.7.3 instead of 2.8.x fixes the problem automagically. So it is very likely that it is not bookdown fault.

cderv commented Nov 28, 2019 • edited

 As @RLesur pointed out, there were some changes in pandoc that may need to be taken into account; I think we need to wait for the issue he opened in pandoc. bookdown is very sensible to change in the html produced by pandoc and if required, we could adjust to new html rendered trying not to break anything. Currently, it is compatible with pandoc 2.7.3 for sure ☺️

dchiu911 commented Nov 29, 2019

 Cross-referencing problem includes Appendix numbering: https://community.rstudio.com/t/bookdowns-gitbook-appendix-header-not-working/45838 Will use pandoc 2.7.3 in the meantime.
added a commit to XiangyunHuang/MASR that referenced this issue Dec 2, 2019
closed this in a07ae07 Dec 5, 2019
added the label Dec 5, 2019

yihui commented Dec 5, 2019 • edited

 Both issues should be fixed now. Thanks a lot for the report and investigation! remotes::install_github(c('rstudio/rmarkdown', 'rstudio/bookdown'))
mentioned this issue Dec 5, 2019

N0rbert commented Dec 5, 2019 • edited

 I'm sorry but cross-reference issue with EPUB --- title: "Cross-reference" site: bookdown::bookdown_site output: bookdown::epub_book: default --- # Part 1 {#p1} ## Chapter 1 {#p1ch1} Some text. ## Chapter 2 {#p1ch2} Here we need to refer to chapter 1 as  \@ref(p1ch1) : see ch. \@ref(p1ch1). remains: > xfun::session_info('bookdown') R version 3.4.4 (2018-03-15) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.6 LTS, RStudio 1.2.5019 Locale: LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=ru_RU.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=ru_RU.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=ru_RU.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=ru_RU.UTF-8 LC_IDENTIFICATION=C Package version: base64enc_0.1.3 bookdown_0.16.4 digest_0.6.23 evaluate_0.14 glue_1.3.1 graphics_3.4.4 grDevices_3.4.4 highr_0.8 htmltools_0.4.0 jsonlite_1.6 knitr_1.26.1 magrittr_1.5 markdown_1.1 methods_3.4.4 mime_0.7 Rcpp_1.0.3 rlang_0.4.2 rmarkdown_1.18 stats_3.4.4 stringi_1.4.3 stringr_1.4.0 tinytex_0.17 tools_3.4.4 utils_3.4.4 xfun_0.11 yaml_2.2.0
added a commit that referenced this issue Dec 5, 2019

yihui commented Dec 5, 2019

 @N0rbert The epub issue should also be fixed now. Thanks!