Discussion topic: flowing footnotes onto other pages

[davidg-sil]

General topic / background.

The basic concept of a footnotes flowing onto another page was in TeX in the 1990s, maybe earlier. Fundamentally, however, we break/disable/ignore the internal (TeX-provided) mechanism in PTXprint.
The internal mechanism relies upon the careful estimation of space left on the page as text and other items are added to the page-list, before lay-up. We have, in one way or another, decided to turn it off and ignored it, not attempting to assign an appropriate size or scale for notes or other inserts. Constraints such as balancing columns rely on a more brute-force approach of re-flowing the entire page until the code accepts the results. By over-collecting input we slow down the code, but ensure that we don't under-fill a page.
Column notes are problematic for the internal mechanism in that they have no impact on the page space until the centre column is full.

Options

1. Implement a mechanism to shave off lines of the final footnote, whatever its class, if the page doesn't fit (assuming it's multiple lines long). One drawback is that this solution preferentially pushes footnotes onto the next page, rather than body-text.

2. Another option is to make a concerted effort to allow TeX to do a better job at properly estimating the page contents. The problem with this is that the re-flow of paragraphed notes is largely incompatible with this mechanism. However, do study bibles with long notes actually use paragraphed notes, or do they, for the reader's sake, use separate notes? Separate notes fit well with the normal TeX way of doing things. Column notes do not.

3. A third option is to pick a particular note class which would be shaved, and put the shaving code into the 'make note box' code. This would allow the user to say "column references don't split, but "\f" notes do.

4. The enhanced version of shaving is to attempt to shave all classes of note. Challenges with this involve some kind of optimisation of 3 or more variables, admittedly with only a few acceptable values.

Open questions

• Is packing more main text on the page an less of the last footnote a big enough problem that it's worth trying approach 2?
• Is the best approach rewriting the code to try footnote-shaving only as a 'last resort' option in some particular circumstance? When?
• Is a mixed-method the best approach - Allow the internal-mechanism to handle separate notes, and the brute-force method for paragraphed notes?
• Shaving footnotes only (currently) works on the last note on the page. What if there are long textual notes and occasional, and unsplittable xrefs?

Current state of work-in-progress

• Shave footnotes with a single breakpoint [proven]
• No shave of footnotes with no breakpoints [proven]
• Handle multiple breakpoints [done]
• Generate multiple breakpoints [done]
• Special handling of last footnote to allow all linebreaks as breakpoints [working]
• Refuse to shave < \LastNoteShaveMin lines onto the next page [done]
• Refuse to shave if doing so will leave < \LastNoteShaveStay [done]
• Refuse to shave last note if it's < \LastNoteShaveShortest lines [TODO]

Un-broken notes

-- nxt page

A break at \fp

-- nxt page

[mhosken]

This looks good. I suggest, though, that we say that only separate notes are shaved and only if a note's height > 4 \baselineskip (where 4 is configurable). Breaking short notes is overly pedantic. We are after the biggies. Likewise breaking at \fp is OK if the currently paragraphed note is > 4 \baselineskip.

[davidg-sil]

Hmm. It'll have to check on the width-before-breaking of the (final) note, then... at least, the current approach is to set the interlinepenalty, widdow penalty etc. as appropriate for the last note on the page, and then see what breaks are possible with \vsplit.

[davidg-sil]

(somewhat belated) update on this topic:
Paragraphed notes now estimate how much space a given note will use on the page (e.g. if the note represents half a line of text, the code counts it as contributing half a line to the height to the page.)

The TeXhacks tab now includes four new 'footnote factors'. These are used to to set the 'knock-on effect' associated with the given type / situation of note - (e.g. for a full width note on a double column page, each line of footnote represents two lines of text. Setting the relevant factor to 2000 would inform TeX of this.) No attempt will be made to add more text to the page (body text or long footnote) than such a calculation would predict, and thus wasted cycles of 'does this much text and that much footnote fit? No, try again' are avoided.
The downside of this is that the estimate of page usage seems to be too large, sometimes reducing the amount of text on the page by a line.

Of the 33 test cases, this is the case in two of them. For the other cases, setting 'correct' factors reduced the processing time by about 10%.

Use of 'correct' factors (rather than deliberately underestimating) is still expected to be valuable, however, for study-Bible style notes.