Pandoc's citeproc gets confused by the presence of footnotes

[wenkokke]

Explain the problem.

Let's take the following document:

---
references:
- type: article-journal
  id: one
  author:
  - family: One
    given: Author
  title: 'The Loneliest Number'
- type: article-journal
  id: two
  author:
  - family: Two
    given: Author
  title: 'Two can be as bad as One'
...

One is the loneliest number [@one[^fn-two]].

[^fn-two]: Two can be as bad as One [@two]

If we run pandoc -C sample.md Pandoc produces the following HTML output:

<p>
  One is the loneliest number
  <span class="citation" data-cites="one">(Two, n.d.)</span>.
</p>
<div id="refs" class="references csl-bib-body hanging-indent" role="list">
  <div id="ref-one" class="csl-entry" role="listitem">
    One, Author. n.d. <span>“The Loneliest Number.”</span>
  </div>
  <div id="ref-two" class="csl-entry" role="listitem">
    Two, Author. n.d. <span>“Two Can Be as Bad as One.”</span>
  </div>
</div>

Notably, Pandoc has produced the text (Two, n.d.) and lost the footnote entirely.
The expected result would be the text (One, n.d.) with the given footnote.

Pandoc version?

$ pandoc --version
pandoc 3.1.6.2
Features: +server +lua
Scripting engine: Lua 5.4
User data directory: ~/.local/share/pandoc
Copyright (C) 2006-2023 John MacFarlane. Web: https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

[jgm]

Write it this way:

One is the loneliest number [@one][^fn-two].

You wouldn't normally want a footnote inside a citation itself.

[jgm]

It does seem to be a bug, however.

[jgm]

Parsing seems correct, so the problem must be in T.P.Citeproc:

    , Cite
        [ Citation
            { citationId = "one"
            , citationPrefix = []
            , citationSuffix =
                [ Note
                    [ Para
                        [ Str "Two"
                        , Space
                        , Str "can"
                        , Space
                        , Str "be"
                        , Space
                        , Str "as"
                        , Space
                        , Str "bad"
                        , Space
                        , Str "as"
                        , Space
                        , Str "One"
                        , Space
                        , Cite
                            [ Citation
                                { citationId = "two"
                                , citationPrefix = []
                                , citationSuffix = []
                                , citationMode = NormalCitation
                                , citationNoteNum = 2
                                , citationHash = 0
                                }
                            ]
                            [ Str "[@two]" ]
                        ]
                    ]
                ]
            , citationMode = NormalCitation
            , citationNoteNum = 2
            , citationHash = 0
            }
        ]
        [ Str "[@one[^fn-two]]" ]

[wenkokke]

Write it this way:

One is the loneliest number [@one][^fn-two].

You wouldn't normally want a footnote inside a citation itself.

This actually renders as:

[One (n.d.)]1

1: fn-two text

[jgm]

Hm, I think there's some bad interaction with reference link syntax. (Inserting a space before the note makes the problem go away.)

[jgm]

Actually, in US style, we'd write the note after the period (I know that not all styles do that):

[@one].[^fn-two]

This works fine.

[jgm]

I've pushed a fix that allows [@one][^fn-two]. to work properly.

[jgm]

I think the issue may be at
https://github.com/jgm/pandoc/blob/main/src/Text/Pandoc/Citeproc.hs#L284-L286
Note the comment

-- assumes we walk in same order as query

I wonder if this assumption is violated in this particular case...