Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-reference problems with Pandoc 2.8 #832

Closed
N0rbert opened this issue Nov 27, 2019 · 13 comments
Closed

Cross-reference problems with Pandoc 2.8 #832

N0rbert opened this issue Nov 27, 2019 · 13 comments
Labels
bug

Comments

@N0rbert
Copy link

@N0rbert N0rbert commented Nov 27, 2019

1. Cross-references are not correctly generated

For simple Rmd-document shown below:

---
title: Cross-reference
site: bookdown::bookdown_site
---

# Part 1 {#p1}

## Chapter 1 {#p1ch1}

Some text.

## Chapter 2 {#p1ch2}

Here we need to refer to chapter 1 as ``` `\@ref(p1ch1)` ```: see ch. \@ref(p1ch1).

I got broken cross-reference with

Rscript -e "bookdown::render_book('cross-reference.Rmd', 'bookdown::html_document2')"

so it looks as follows:

broken

(please note "?? in output above). All often-used writers such as HTML, EPUB, ODT and DOCX are affected. But the PDF is not affected.

Note: this is repost of jgm/pandoc#5941 and the Pandoc author said that this is not their issue. The problem is caused only by pandoc 2.8. As all other related components remain the same.

2. Figure, table, example and equation numbering changed from two-level to single-level in HTML output

For simple Rmd-document shown below:

---
title: Item numbering
site: bookdown::bookdown_site
---

# Figure

![(\#fig:md-logo) Markdown logo](Markdown-mark.svg.png)

Link: see Fig. \@ref(fig:md-logo) ...

# Table

Table: (\#tab:results) Demo table

| **Head L** | **Head R** |
|------------|------------|
| r1, c1     | r1, c2     |
| r2, c1     | r2, c2     |

Link: see Table \@ref(tab:results) ...

# Equation

\begin{equation}
f\left(k\right) =\binom{n}{k} p^k\left(1-p\right)^{n k}
(\#eq:binom)
\end{equation}


Link: see equation \@ref(eq:binom) ...

# Example

```{example, simple-code}
Simple code
```
```
int main() {
  return 0;
}
```

Link: see example \@ref(exm:simple-code) ...

I got single-level numbering of objects which is not expected for me:

Rscript -e "bookdown::render_book('numbering.Rmd', 'bookdown::html_document2')"

ends with rendering below:

wrong numbering

(with pandoc 2.7.3 numbering had two-levels)

Note: this is a repost of jgm/pandoc#5942. The problem is caused only by pandoc 2.8. As all other related components remain the same. The EPUB writer is affected too.

My version info:

> xfun::session_info('bookdown')
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS, RStudio 1.2.5019

Locale:
  LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=ru_RU.UTF-8       
  LC_COLLATE=en_US.UTF-8     LC_MONETARY=ru_RU.UTF-8    LC_MESSAGES=en_US.UTF-8   
  LC_PAPER=ru_RU.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
  LC_TELEPHONE=C             LC_MEASUREMENT=ru_RU.UTF-8 LC_IDENTIFICATION=C       

Package version:
  base64enc_0.1.3 bookdown_0.16   digest_0.6.23   evaluate_0.14   glue_1.3.1      graphics_3.4.4 
  grDevices_3.4.4 highr_0.8       htmltools_0.4.0 jsonlite_1.6    knitr_1.26.1    magrittr_1.5   
  markdown_1.1    methods_3.4.4   mime_0.7        Rcpp_1.0.3      rlang_0.4.2     rmarkdown_1.18 
  stats_3.4.4     stringi_1.4.3   stringr_1.4.0   tinytex_0.17    tools_3.4.4     utils_3.4.4    
  xfun_0.11       yaml_2.2.0     
> 

By filing an issue to this repo, I promise that

  • I have fully read the issue guide at https://yihui.org/issue/.
  • I have provided the necessary information about my issue.
    • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
    • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('bookdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/bookdown').
    • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
  • I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

@yihui

This comment has been minimized.

Copy link
Member

@yihui yihui commented Nov 27, 2019

Thanks for the report, but I probably won't have time to investigate this in the near future. Any particular reason why you need to use the very latest version of Pandoc instead of the version bundled in RStudio?

@yihui yihui changed the title problem with pandoc 2.8: cross-references inside document are broken Cross-reference problems with Pandoc 2.8 Nov 27, 2019
@RLesur

This comment has been minimized.

Copy link
Collaborator

@RLesur RLesur commented Nov 28, 2019

For bookdown::html_document2, I think problem 1 comes from new attributes in headings.
With Pandoc 2.8 and 2.8.0.1:

pandoc -N --section-divs -t html4 <<< "# Hello"

gives

<div id="hello" class="section level1">
<h1 number="1"><span class="header-section-number">1</span> Hello</h1>
</div>

I guess this regex should be modified

reg_num = '^(<h[12]><span class="header-section-number">)([.A-Z0-9]+)(</span>.+</h[12]>).*$'

BTW, the goal of Pandoc 2.8 was to write:

<div id="hello" class="section level1">
<h1 data-number="1"><span class="header-section-number">1</span> Hello</h1>
</div>

I've opened an issue in Pandoc to warn about this invalid number attribute.

@N0rbert

This comment has been minimized.

Copy link
Author

@N0rbert N0rbert commented Nov 28, 2019

Any particular reason why you need to use the very latest version of Pandoc instead of the version bundled in RStudio?

I use GitLab CI/CD pipeline with latest pandoc to have fresh functionality. So it is head-less mode without RStudio. I'll stay with pandoc 2.7.3 to prevent unpredictable behaviour.

I think problem 1 comes from new attributes in headings.

Thanks, friends. I appreciate your efforts :)
I'll be very happy if we have stable bookdown<->pandoc experience.

@yihui

This comment has been minimized.

Copy link
Member

@yihui yihui commented Nov 28, 2019

@N0rbert I don't know about GitLab's CI, but on Travis, you can easily specify the Pandoc version: https://docs.travis-ci.com/user/languages/r/

@RLesur Thanks for the tip! I think I should be able to fix the first problem soon.

@N0rbert

This comment has been minimized.

Copy link
Author

@N0rbert N0rbert commented Nov 28, 2019

@yihui Thank you for info about Travis!
How can I help with debugging the second problem?

@RLesur

This comment has been minimized.

Copy link
Collaborator

@RLesur RLesur commented Nov 28, 2019

I was wrong about data-number, see jgm/pandoc#5944 (comment). If Pandoc is fixed and removes the extra number attribute, there will be no need for a fix in bookdown for the first problem.

@RLesur

This comment has been minimized.

Copy link
Collaborator

@RLesur RLesur commented Nov 28, 2019

@N0rbert If you are interested in, I also used GitLab CI to debug the first problem. My repo is here: https://gitlab.com/RLesur/bookdown-issue832.

@N0rbert

This comment has been minimized.

Copy link
Author

@N0rbert N0rbert commented Nov 28, 2019

@RLesur I'm interested, but I currently I can't get all the debugging information on my system because of lack of knowledge.
In YAML currently I'm using

output:
  bookdown::html_document2:
    pandoc_args: ['--trace', '--verbose', '--log=pandoc.log']
    clean: false
    keep_md: true

and build with

Rscript -e "bookdown::render_book('cross-reference.Rmd', 'bookdown::html_document2', clean=FALSE)"

but it does not highlight the problem.

@yihui
Even after looking thorough on

I can't understand which application converts \@ref(...) to html-links. How can we debug this?
How can I run all the steps manually one-after-one?
How should we determine which of bookdown and/or pandoc is really causes the problem?

What I can see from user point-of-view - installing pandoc 2.7.3 instead of 2.8.x fixes the problem automagically. So it is very likely that it is not bookdown fault.

@cderv

This comment has been minimized.

Copy link
Collaborator

@cderv cderv commented Nov 28, 2019

As @RLesur pointed out, there were some changes in pandoc that may need to be taken into account;
I think we need to wait for the issue he opened in pandoc. bookdown is very sensible to change in the html produced by pandoc and if required, we could adjust to new html rendered trying not to break anything.
Currently, it is compatible with pandoc 2.7.3 for sure ☺️

@dchiu911

This comment has been minimized.

Copy link

@dchiu911 dchiu911 commented Nov 29, 2019

Cross-referencing problem includes Appendix numbering:
https://community.rstudio.com/t/bookdowns-gitbook-appendix-header-not-working/45838

Will use pandoc 2.7.3 in the meantime.

@yihui

This comment has been minimized.

Copy link
Member

@yihui yihui commented Dec 5, 2019

Both issues should be fixed now. Thanks a lot for the report and investigation!

remotes::install_github(c('rstudio/rmarkdown', 'rstudio/bookdown'))
@mnazarov mnazarov mentioned this issue Dec 5, 2019
3 of 3 tasks complete
@N0rbert

This comment has been minimized.

Copy link
Author

@N0rbert N0rbert commented Dec 5, 2019

I'm sorry but cross-reference issue with EPUB

---
title: "Cross-reference"
site: bookdown::bookdown_site
output:
  bookdown::epub_book: default
---

# Part 1 {#p1}

## Chapter 1 {#p1ch1}

Some text.

## Chapter 2 {#p1ch2}

Here we need to refer to chapter 1 as ``` `\@ref(p1ch1)` ```: see ch. \@ref(p1ch1).

remains:

epub output


> xfun::session_info('bookdown')
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS, RStudio 1.2.5019

Locale:
  LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=ru_RU.UTF-8       
  LC_COLLATE=en_US.UTF-8     LC_MONETARY=ru_RU.UTF-8    LC_MESSAGES=en_US.UTF-8   
  LC_PAPER=ru_RU.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
  LC_TELEPHONE=C             LC_MEASUREMENT=ru_RU.UTF-8 LC_IDENTIFICATION=C       

Package version:
  base64enc_0.1.3 bookdown_0.16.4 digest_0.6.23   evaluate_0.14   glue_1.3.1      graphics_3.4.4 
  grDevices_3.4.4 highr_0.8       htmltools_0.4.0 jsonlite_1.6    knitr_1.26.1    magrittr_1.5   
  markdown_1.1    methods_3.4.4   mime_0.7        Rcpp_1.0.3      rlang_0.4.2     rmarkdown_1.18 
  stats_3.4.4     stringi_1.4.3   stringr_1.4.0   tinytex_0.17    tools_3.4.4     utils_3.4.4    
  xfun_0.11       yaml_2.2.0     
yihui added a commit that referenced this issue Dec 5, 2019
@yihui

This comment has been minimized.

Copy link
Member

@yihui yihui commented Dec 5, 2019

@N0rbert The epub issue should also be fixed now. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.