An option to embed footnotes earlier than on the end of document

[Mekk]

It would be great, if footnote extension allowed one to decide where should the footnotes text be placed, allowing (depending on some config param) one to:

• put all footnote definitions on the end of the document (default, the way things work currently)
• put footnote definitions on the end of the section footnotes are used (= show all referred footnotes before the next section starts) { extra option - set section level }
• put footnote definitions after the current paragraph (= show all refererred footnotes once current paragraph/table/block/whatever is finished)
• expand them inline where referred, just wrap with some CSS-identifiable span

Rationale:

a) (my main use-case) On ebook-reader (while viewing EPUB file) navigating to the footnote, and back, tend to be slow and (on non-touch readers) sometimes require troublesome navigation. Having footnote rendered below the current paragraph (likely styled with smaller font) would make much nicer user experience.

b) Also on webpages it may make better presentation if footnotes are close to the text they refer to (here some javascript instrumentation may make it possible to dynamically show them on mouseover or mouseclick)

[Mekk]

I considered hacking some code, but I lack the knowledge what is the best way to locate places like "end of section" or "end of paragraph" (and at what stage should it happen). If you could make some suggestions regarding this, I am ready to work on the issue.

[waylan]

I will note that our footnote extension is intended to be a direct
clone of the footnote functionality found in PHP Markdown
Extra.
That said, I can see your use case. In fact, I've recently been
annoyed by less that ideal footnotes in epub readers and have been
thinking about similar solutions.

However, I'm not sure that such a solution belongs in the markdown
library. Perhaps this would be more appropriate as a third party
extension. You certainly could fork the existing extension as a
starting point, assuming licences are compatible. You might want to
check other implementations and/or search the markdown
list for
existing solutions and copy their syntax. I seem to remember this
being brought up before.

By biggest problem with your suggestion is placing the footnotes after
whatever paragraph they are referenced from. This seems to me to be
much more appropriate for javascript than Markdown. Regarding
Javascript solutions,
here's
an already working popup library (also as a browser
extension).
I realize it doesn't address the non-touch screens but it does
understand the syntax so it could work as a starting point.

Hope that helps. If you need any help just ask.

Waylan

[Mekk]

Just a few comments to comments.

\1. Whatever origins python-markdown has, it now lives on it's own. In particular, I am yet to see PHP .epub generator, while calibre is more and more popular (and python markdown is it's best option for formatted plain text). In general, while it is of course sensible to keep source-text-level compatibility, adding extra generator options does not break things (and, after all, python markdown and perl markdown are not script-option-level compatible and for PHP there is no script at all).

My suggestion is to add an extra option which influences the output organization but preserves the same source syntax. Just like - say - CodeHiliteExtension allow one to set css_class and decide whether line numbers are to be shown, or not, FootnotesExtension could allow one to setup the footnotes location.

\2. Forking is troublesome as this is not only a matter of using the tool as such, but using it in the ecosystem. Keeping my main use-case, to use forked extension in calibre I would need to persuade calibre maintainer to drop your version in favor of mine (in case it is written, of course), and then duplicate maintaining efforts… The same for other markdown-using tools. Having new option in existing markdown means just writing the appropriate switch on one of the configuration fields.

\3. JavaScript is not always an option - for example most ebook-readers does not handle javascript at all. And even with javascript present, having footnotes nearby makes it easier to script the nice presentation (in most cases it will suffice to hide/unhide the block of given id, no need to find it and move it around)

Well, maybe I will try to make some proof of concept.

My main technical question is the same as earlier: what is the best way to find places like "end of the section" or "end of paragraph" (I guess it is possible to walk the tree on treeprocessor level but it would be nicer if I could locate appropriate places before HTML is generated)

[waylan]

On Tue, Nov 22, 2011 at 3:32 AM, Marcin Kasperski
reply@reply.github.com
wrote:

Just a few comments to comments.

1. Whatever origins python-markdown has, it now lives on it's own. In particular, I am yet to see PHP .epub generator, while calibre is more and more popular (and python markdown is it's best option for formatted plain text). In general, while it is of course sensible to keep source-text-level compatibility, adding extra generator options does not break things (and, after all, python markdown and perl markdown are not script-option-level compatible and for PHP there is no script at all).

No, IMO each markdown implementation should strive to implement a
parser that outputs essentially the same given an input. There are
numerous implementations in many different languages and when one
implementation is different, it divides Markdown's (the markup
language) user base and adds confusion. Go ahead and search the list
archives I linked earlier and you will see multiple instances of that
occurring. It is never a good thing.

My suggestion is to add an extra option which influences the output organization but preserves the same source syntax. Just like - say - CodeHiliteExtension allow one to set css_class and decide whether line numbers are to be shown, or not, FootnotesExtension could allow one to setup the footnotes location.

True, but the CodeHilite extension was the second extension I ever
wrote (before I had any official involvement in Python-Markdown) and
today I would do things quite different; namely less options. The more
we add to the core code base, the more we have to maintain. When that
includes nonstandard features (which we may not need or use) it adds
to our maintenance work. Of course, when those features are standard
and/or make sense (filling a real and common need), then we're open to
the possibility of including them as non-default options (I'm
undecided if your situation qualifies). However, in most cases, the
need is one-of-a-kind and therefore it makes more sense for the person
with the need to implement and maintain their own extension.

In fact, that is the primary purpose of the public extension API.
Anyone can use our engine to implement any syntax they want to fit
their needs without relying on us to update the core code. That really
is the best for everyone.

To convince me that your feature request should be added to the core,
I'm probably going to need to see some prior art. Is this even a
problem that others are having? What have other implementations done
to address this issue? You might want to search the list archives (or
ask their) and/or check out how any other implementations may have
addressed the issue (see this incomplete
list](http://babelmark.bobtfish.net/) ).

1. Forking is troublesome as this is not only a matter of using the tool as such, but using it in the ecosystem. Keeping my main use-case, to use forked extension in calibre I would need to persuade calibre maintainer to drop your version in favor of mine (in case it is written, of course), and then duplicate maintaining efforts… The same for other markdown-using tools. Having new option in existing markdown means just dropping the appropriate switch on one of the configuration fields.

I'm not familiar with Calibre's plug-in API, but if it allows you to
set markdown's config settings, then you should be able to use your
own extension. Rather than passing in a string (usually the extension
name) simply pass in an instance of your extension (an instance of a
subclass of markdown.extensions.Extension). In fact, it doesn't even
need to be in a separate module from your plug-in. If Calibre won't
allow this but does allow you to pass in strings as names of
extensions, then this is an issue with Calibre, not Python-Markdown.

BTW, if it was me, I would probably not fork the footnote extension,
but build a separate extension which simply swaps out the existing
extensions Treeprocessor for a new one. Just make sure the existing
footnote extension is loaded first, which should be easy as the
extensions keyword (of markdown.Markdown) expects a list and
extensions are loaded in the order they occur in that list.

1. JavaScript is not always an option - for example most ebook-readers does not handle javascript at all. And even with javascript present, having footnotes nearby makes it easier to script the nice presentation (in most cases it will suffice to hide/unhide the block of given id, no need to find it and move it around)

Well, maybe I will try to make some proof of concept.

My main technical question is the same as earlier: what is the best way to find places like "end of the section" or "end of paragraph" (I guess it is possible to walk the tree on treeprocessor level but it would be nicer if I could locate appropriate places before HTML is generated)

Walking the tree seems like the most reasonable way to me. That's
probably how I would do it. Unfortunately, ElementTree elements don't
know about their parents like JS DOM objects do. That just makes
walking the tree a little trickier, but it is still possible and well
documented in the ElementTree docs.

---

\X/ /-\ `/ |_ /-\ ||
Waylan Limberg

[waylan]

I didn't close this initially because I wanted to give the idea the opportunity to grow on me. After 4 months it hasn't. In fact, I'm only more sure about my position, so I'm closing this issue. This definitively belongs as a third party extension.

Regarding the need to split a document up into sections, a couple of third party extensions are listed on the wiki that do that. Might want to start there.

[Mekk]

Considering your earlier opinion I resolved the issue by migrating to textile.