pptx and xlsx readers: speaker notes, charts, hidden slides, numbers and dates

[nachogarcia]

We use the new pptx and xlsx readers to turn academic slide decks and
workbooks into Markdown for review. Thanks for adding them! Several plain
bugs are already filed as issues, each with a PR:

• Xlsx reader: fails on workbooks written by openpyxl and pandas (absolute relationship targets) #12013 / Xlsx reader: resolve package-absolute relationship targets #12018: xlsx files written by openpyxl or pandas fail (absolute relationship targets)
• Xlsx reader: inline strings dropped; spaces added inside rich-text strings #12014 / Xlsx reader: read inline strings, and rich text as written #12019: xlsx inline strings dropped; spaces added inside rich text
• one far-away xlsx cell makes conversion unbounded: moved to xlsx reader: keeping sheet tables bounded by their content #12024, as the first fix wasn't general enough
• Pptx reader: spaces added between text runs; line breaks dropped #12016 / Pptx reader: don't add spaces between text runs #12021: pptx text runs joined with spaces; line breaks dropped
• Pptx reader: content of grouped shapes is dropped #12017 / Pptx reader: read the shapes inside group shapes #12022: pptx group shapes dropped

A few other changes add behaviour, so I'd like to check before opening PRs.

1. pptx: speaker notes. The reader never opens the notesSlide parts.
On four CC-BY academic decks from Zenodo, which hold 29 non-empty notes
between them, it reads none. I have a branch, now #12030, that reads each slide's notes
into a ::: notes div at the end of the slide. That is the form the pptx
writer turns into speaker notes, so pptx → Markdown → pptx keeps them.
It follows the slide's own relationship, because notesSlideN.xml often
isn't slide N's (in one of those decks, 32 of 33 numbers differ). It also
skips notes parts with no text.

Question: should notes be read by default? In HTML or docx output they
show up as an ordinary div. If you'd rather make them opt-in, I can put
them behind a reader extension.

2. pptx: charts. A chart currently becomes the paragraph
[Graphic: other: http://schemas.openxmlformats.org/drawingml/2006/chart].
The chart part carries cached values for every series (c:cat and c:val
with their strCache/numCache), so the reader could emit the chart's
title and a table with one column per series, keeping the placeholder
when the chart part can't be read. Would you take that?

3. pptx: hidden slides. Slides with show="0" are read like any
other, with nothing to mark them. Leave that as it is, mark them (for
example with a hidden class on the slide's header), or skip them?

4. xlsx: numbers. Values are rendered with Haskell's show: 0.05
becomes 5.0e-2, 12345678 becomes 1.2345678e7, 21 becomes 21.0,
and booleans become 1.0/0.0. I'd propose rendering them the way Excel's
General format does: integers without .0, no exponent for ordinary
magnitudes, and TRUE/FALSE. Would you treat that as a bug fix?
Or do you want a wider change that honours each cell's number format
(decimals, percentages, thousands separators)?

5. xlsx: dates. No number formats are parsed, so a date comes out as
its serial number (45231.0). The proposal is to read numFmts and
cellXfs. Cells with a date or time format (built-in ids 14–22 and
45–47, or a custom format with date tokens) would be rendered as ISO 8601,
honouring the workbook's 1904 date system flag.

Happy to adjust any of these to fit how you'd like the readers to evolve.

[jgm]

1. I'd say notes should be read back into the form that the reader itself consumes, as you describe.
2. I don't really understand what a "series" is here. Maybe you could give an example?
3. Don't know. Adding a hidden class seems reasonable.
4. Yes, this is an improvement. Honoring the number format would be even better if it doesn't add too much complexity.
5. Making the dates look like dates would be a definite improvement!

[nachogarcia]

Thanks! Taking these in turn:

1. I'll open the speaker-notes PR as described.

2. A "series" is one set of values plotted against the chart's categories: one set of bars, or one line. A pptx chart part stores the last values it displayed for each series, alongside the reference into its embedded workbook. So the data can be read without opening that workbook. For example, a bar chart from one of our test decks has one series, "Mean growth", over four categories:

   <c:ser>
     <c:tx>…<c:v>Mean growth</c:v>…</c:tx>
     <c:cat>…<c:v>Debt/GDP below 30%</c:v> <c:v>30 to 60%</c:v> <c:v>60 to 90%</c:v> <c:v>90% and above</c:v>…</c:cat>
     <c:val>…<c:v>4.1</c:v> <c:v>2.8</c:v> <c:v>2.8</c:v> <c:v>-0.1</c:v>…</c:val>
   </c:ser>
   

   Today that slide reads as [Graphic: other: http://schemas.openxmlformats.org/drawingml/2006/chart]. The proposal is to read it as a table, captioned with the chart's title if it has one:

     Category             Mean growth
     -------------------- -------------
     Debt/GDP below 30%   4.1
     30 to 60%            2.8
     60 to 90%            2.8
     90% and above        -0.1
   

   A chart with two series, say growth in 2019 and in 2020 per category, would get a third column. The chart type and styling would be lost, but the numbers would survive.

3. I'll add a hidden class to the header of hidden slides.

4 and 5. I'll start with General-format numbers and dates as separate PRs. Then I'll see how far honouring the cell's own number format can go without much complexity.