Footnote in footnote (markdown)

[Kedrigern]

Hi, I have text like this:

Lorem[^1] ipsum. 

[^1]: Footnote 1 text[^A]

[^A]: Footnote A text

and I would like to convert them into LaTeX.

I expect:

Lorem\footnote{Footnote 1 text\footnote{Footnote A text}} ipsum.

but I get:

Lorem\footnote{Footnote 1 text{[}\^{}A{]}} ipsum.

Replace letter (A) by number does not help.

I try markdown, markdown_github, markdown_mmd, markdown_phpextra, markdown_strict.

[jgm]

Pandoc's markdown doesn't support footnotes inside footnotes (which are an abomination anyway!).

[Kedrigern]

For example ReText editor (use python-markups, python-markdown) has this functionality.

I work with very complex scientific texts. Sometimes footnote inside footnote is important. For example translator's note inside author's note. Ofcourse in simple blogpost this is not necessary.

[Kedrigern]

BTW: MultiMarkdown inline footenotes does not work.

[jgm]

+++ Ondřej Profant [Apr 05 15 15:04 ]:

BTW: MultiMarkdown inline footenotes does not work.

Sigh. Looks like Fletcher added inline footnotes in 2013. Why couldn't he have used the same syntax as pandoc's inline footnotes, which we've had since 2007? Instead he uses a slightly different syntax, contributing to the mess of proliferating markdown syntaxes...

[jgm]

+++ Ondřej Profant [Apr 05 15 14:25 ]:

For example ReText editor (use python-markups, python-markdown) has this functionality.

I work with very complex scientific texts. Sometimes footnote inside footnote is important. For example translator's note inside author's note.

Footnotes inside footnotes don't work well with many of the output
formats pandoc supports. IN LaTeX, for example, you'd need to use an
external package to do this:
http://tex.stackexchange.com/questions/23157/footnote-to-a-footnote

So, I'm not inclined to support this.

[lierdakil]

Not arguing that nested footnotes should be supported, but some manual
messing about with \footnotemark and \footnotetext should probably work
with LaTeX output. No need for external packages. It is, of course, ugly
and unintuitive.
​

2015-04-06 1:59 GMT+03:00 John MacFarlane notifications@github.com:

+++ Ondřej Profant [Apr 05 15 14:25 ]:

For example ReText editor
(use python-markups,
python-markdown) has this
functionality.

I work with very complex scientific texts. Sometimes footnote inside
footnote is important. For example translator's note inside author's note.

Footnotes inside footnotes don't work well with many of the output
formats pandoc supports. IN LaTeX, for example, you'd need to use an
external package to do this:
http://tex.stackexchange.com/questions/23157/footnote-to-a-footnote

So, I'm not inclined to support this.

—
Reply to this email directly or view it on GitHub
#2053 (comment).

[Jmuccigr]

Footnotes within footnotes? Who does that? Why do you even need it? The point of having a footnote is to get the note text out of the way when reading. If you're down in the footnote already, that functionality is unnecessary.

I think all I've ever seen is something like an inline ref: "See n. 5."

[lierdakil]

Footnotes within footnotes can be used to a comedic effect for literary
fiction, and as a way to put translator's notes w.r.t. a pre-existing
footnote. So it's not entirely pointless. However, it may be not worth the
considerable effort.

2015-04-06 21:00 GMT+03:00 John Muccigrosso notifications@github.com:

Footnotes within footnotes? Who does that? Why do you even need it? The
point of having a footnote is to get the note text out of the way when
reading. If you're down in the footnote already, that functionality is
unnecessary.

I think all I've ever seen is something like an inline ref: "See n. 5."

—
Reply to this email directly or view it on GitHub
#2053 (comment).

[Jmuccigr]

Interesting. Can you point me to something using it?

[lierdakil]

For one, Sir Terry Pratchett extensively used nested footnotes in his
Discworld series to an arguably great effect. I'm also pretty sure that
Chicago Manual of Style allows for nested footnotes in some particular
cases, but I can't quote from it, since I don't have a copy at hand.

2015-04-06 21:52 GMT+03:00 John Muccigrosso notifications@github.com:

Interesting. Can you point me to something using it?

—
Reply to this email directly or view it on GitHub
#2053 (comment).

[Jmuccigr]

Googling a bit, most of what I find for nested footnote is on TeX pages (along with lots of "don't do it"). I didn't see anything for Chicago Manual, except for referring to one footnote within another, not creating a footnote within a footnote (which, again, I think is unnecessary, except for comic effect).

And apparently we should have cited this right away: https://xkcd.com/1208/

[lierdakil]

There are, however, a sizeable amount of tutorials on how to achieve nested footnotes in LaTeX and/or Word, so I gather that creating footnote labyrinths is a somewhat popular pastime. Never felt the need to use them myself though.

[Kedrigern]

As I have said. For example you need translater footnotes (or endnotes) in translated text. You have regular author footnote and translator need to tell you something.

Mentioned sir Terrry Pratchett using footnotes often and Czech tranlate contains many translator footnote in author footnote. (My experince)

[jjaderberg]

Danielewski's House of Leaves is another example from fiction of nested footnotes, but there they are not used for comic effect but are an indispensable structural component for telling a complicated story. The book is perhaps not a model of sensible typography, but it is very effective and was quite popular.

I'm not sure I see the need for these examples, however. I'm quite willing to take @Kedrigern at his word, and will not try to convince him that he doesn't need nested notes, really, or that he ought not to use them; I trust him to be the judge of that ;-)

With regards to what @jgm says about needing an external package for this, my impression is that this is true for most anything one does in latex. Looking at the default latex template for pandoc it seems several external packages are already relied upon (hyperref, fontspec, amsmath etc). I use bigfoot for notes, and have a conditional clause in my template that includes that package if a certain parameter is set in the YAML header of my document. It defines one or more new footnote series that I can use with latex commands in my markdown source. This is not exclusively for nested notes, but also very useful when one wants a different series of notes on the first page–dagger and asterisk symbols for author and 'thank you' notes, for instance–or elsewhere. They do work just fine, however, inside the footnotes of the default series that I type with pandoc-markdown syntax.

I do use nested notes a fair bit in my research. When I study an article or book closely I often recreate it as markdown/latex/pdf files. The default series of footnotes are then taken up by the notes from the original. I critique or expand arguments in the original work within my own foot- and margin notes, and insert hypertext links to other articles that I have "imported" or to my own writings, and sometimes this means notes within notes. I would not be able to work this way without nesting.

I am not necessarily pushing for support for nested notes in pandoc-markdown, however, if this seems complicated or undesirable for whatever reason–I can do what I want with latex commands in the markdown text. I'm quite content with nested notes being considered a marginal use case without any special support in pandoc-markdown. I do want to challenge the assumption that there is no meaningful use case for nested notes, and to suggest that it is possible already to achieve this with the bigfoot latex package and some minor tweaking to the template.

[Kedrigern]

@jjaderberg 👍 Thanks you, exactly what I thought.

Or maybe we can consider support for endnotes.

Footnote, endnote and citation are usual in research.

[tarleb]

The change may require additional work in pandoc's writersreaders to prevent infinite loops. Given the limited use case and the existence of a simple workaround, I'd suggest to close this.

[lierdakil]

@tarleb, I don't see how nested footnotes could lead to infinite loops. Like, at all. And I'm reasonably familiar with Pandoc's code. So this argument doesn't seem valid. Besides, workaround only applies to LaTeX output, and won't work at all if one targets multiple output formats.

Primary concern with nested footnotes isn't parsing, but output. While it's reasonably simple with some output formats, LaTeX and OOXML (docx) have quite a bit of quirks wrt nested footnotes (latter having trouble with footnotes as is). And while it's certainly possible to work around that, it would introduce a maintenance overhead, which isn't ideal.

[tarleb]

How to DoS pandoc using footnotes:

• replace lines 1842 ff. in Text.Pandoc.Reader.Markdown with just B.notes <$> content.
• Recompile.
• Run with echo "[^1]\n\n[^1]: hello[^1]" | pandoc -f markdown -t native.
• Witness pandoc eating away all available memory.

You are right that the workaround only works with LaTeX, maybe HTML. It's not ideal. But there are many other features for which this is true. Some of those features should be added, others might just be too much effort and create too much complexity. I believe this issue falls into the latter category.

[lierdakil]

@tarleb, right, sorry. It's been a while and for some reason my memory insisted that Pandoc uses two-pass parser, while it, in fact, doesn't. That might be related to the fact that I'm using two-pass approach in pandoc-crossref and my mind somehow mixed up the two >_> Sorry again.

That said, this highlights a bit of a problem with footnotes in general -- every reference duplicates definition. All fine and dandy if there's only one reference per footnote, and a horrible mess if there are several. Consider this for example:

$ echo -e "[^1][^1]\n\n[^1]: footnote1" | pandoc -t markdown
[^1][^2]

[^1]: footnote1

[^2]: footnote1

Fixing that would likely require some changes to AST though, and it's arguable if it's necessary for footnotes (as opposed to endnotes, although depending on output format, difference can be blurry)

[tarleb]

One could argue that Pandoc's use of the F reader monad results in a two-pass read. Kind of. So you were not wrong per se.

I seem to remember some discussion regarding the footnote problem, talking about the exact issues you mention, though I cannot seem to find it atm. It's reasonable to interpret the issue at hand as merely a symptom of a deeper problem with the way footnotes are handled. Thanks for highlighting it.

[gwern]

One thing I would note about recursive footnotes is that right now they are handled badly. They emit no warning, they drop footnotes unpredictably, and they generate, for HTML5 output, weird output with duplicate IDs (I am not sure if that is explicitly invalid HTML, but it's not good).

An example defining four footnotes:

test[^footnote-1]

[^footnote-1]: test 2^[Second footnote, inline.] but note[^footnote-3]

[^footnote-3]: What does this do?^[Fourth footnote, inline.]

compiles to

<article>
  <p>test<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a></p>
  <section class="footnotes" role="doc-endnotes">
  <hr />
  <ol>
  <li id="fn1" role="doc-endnote"><p>test 2<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a> but note[^footnote-3]<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
  </ol>
  </section>
</article>

Which HTML Tidy notes:

line 43 column 44 - Warning: <a> anchor "fnref1" already defined

Since it's defined both in the body and in endtext where the first footnote is.
(I noticed a Tidy warning on one of my pages, wondered how such a warning could possibly happen when Pandoc handled all the footnote generation, and after tracking it down, realized what was going on: I'd split out a section as a named footnote, but hadn't noticed that section already had a footnote, inline.)

The final rendered output shows that, surprisingly, the first footnote gets rendered, a superscript inside it gets rendered for footnote 2, but then it points to itself so it has a rendered ID/superscript but not valid link or body, while footnote 3 appears inline as a Markdown fragment ID (with no superscript, link, or body), and footnote 4 disappears entirely.

Given that there doesn't seem to be any prospect of implementing recursive footnotes within footnotes and doing anything with footnotes inside footnotes leads to bad and confusing partial outputs, it seems like trying to render the footnotes is a bug, or at least perhaps it should be an error or warning?

[jgm]

Yes, it would probably make sense to disable the footnotes extension when we're parsing notes. Or better, leave it enable but change the behavior, so that a warning is produced but no note.

[St-Ex]

I'd like to add another impact of footnote in footnote.

If you have a footnote containing a link and you use links-as-notes variable. It creates a reference to a non-existing link.

A paragraph with a reference[^reference] where the reference formatted content in footnotes is bad.

[^reference]: The [Pandoc](https://pandoc.org/MANUAL.html) website

A paragraph with a reference\footnote{The
  \href{https://pandoc.org/MANUAL.html}{Pandoc} website} where the
reference formatted content in footnotes is bad.

Which then return

However, it exists a good workaround for this case : Using link label equal to link target produce great output

A paragraph with a reference[^reference] where the reference formatted content in footnotes is bad.

[^reference]: The [https://pandoc.org/MANUAL.html](https://pandoc.org/MANUAL.html) website

[brainchild0]

How to DoS pandoc using footnotes:

• replace lines 1842 ff. in Text.Pandoc.Reader.Markdown with just B.notes <$> content.

• Recompile.

• Run with echo "[^1]\n\n[^1]: hello[^1]" | pandoc -f markdown -t native.

• Witness pandoc eating away all available memory.

@tarleb: In the example you provide, I find a cycle (the footnote refers directly to itself). Would the same operation terminate if the references had no cycles? Since the footnote definitions in the Markdown text appear in a list, we could require that each nested footnote reference appear earlier in the list than the definition of that footnote (the same as how footnote references in body text usually appear before the footnote definitions list). This constraint prevents any cycles. Then, can we avoid infinite loops, such as the one in your example?

[tarleb]

@brainchild0 you may not be aware of it, but the majority of pandoc's core developers are non-native speakers. The flowery language of your comments may be nice to read in a non-technical context; here however, it just makes our lives more difficult. Having to first wade through and decipher a wall of text makes my life harder and also makes it less likely for me to consider a comment. Please try to keep the comments short and to the point and avoid unnecessary contextualization (it is usually save to assume people to be able to do that themselves).

[brainchild0]

@tarleb Sure I understand. I revised the comment into plain language. I hope the result is closer to what you and others prefer. I would be happy to further revise, clarify, simplify, or elaborate, however may be helpful.