xlsx reader: keeping sheet tables bounded by their content

[nachogarcia]

The xlsx reader turns each sheet into one table covering every row and column between the sheet's outermost cells. The table's size therefore isn't bounded by what the file contains:

• A 2.4 kB workbook with a small table and one value in XFD1048576, the last cell of a sheet, asks for 16384 × 1048576 cells and doesn't finish.
• Empty cells that only carry a style count as well, and spreadsheet programs often leave those far from the data.

I opened #12020 to keep only the rows and columns that hold a non-blank cell. That fixes the stray-cell case but not the general one. The table is then (rows with content) × (columns with content), which can still be quadratic in the content. A 235 kB workbook with one full row (16384 values) and 20000 values down column A still needs about 330 million cells, and didn't finish in 60 s. So I closed #12020 and #12015. Since anyone can hand pandoc an .xlsx file, I'd like to agree on the approach before a new PR.

Options I can see:

1. Keep only non-blank rows and columns (Xlsx reader: build tables from the non-blank rows and columns #12020). This fixes stray and styled-empty cells, but drops blank separator rows. It is still quadratic in the worst case, so it would need one of the options below as well.
2. Cap the number of cells in a table. Above a limit (a fixed number, or a multiple of the number of non-blank cells), truncate and report a warning. Output stays a table, but a legitimate large sheet could be cut.
3. Switch representation when a table would be mostly empty. If the table would be more than k times larger than the number of non-blank cells, emit the sheet differently, for example as a list of A1: value entries. Output then grows with the content in every case, but those sheets look different from the others.
4. Split a sheet into its separate blocks of cells, those separated by fully blank rows and columns, with one table per block. This matches sheets that hold several tables, but doesn't bound the worst case by itself either.

Which would you prefer? My inclination is 1 + 3, so that output always grows with the content. Or 1 + 2, if you'd rather every sheet stay a table.

[jgm]

I really don't know. It would be good if people who use pandoc with xslx chime in here.

Does this problem arise in real-world cases?

[nachogarcia]

Not that I've seen. All 73 real-world workbooks I tested convert fine on main, and none has used cells anywhere near a sheet's limits. The case only comes from a crafted file: a 2.4 kB workbook with one value in the last cell of the sheet. So it matters for services that convert untrusted uploads with pandoc. I'll leave this open in case xlsx users have views on how tables with gaps should look.

[jgm]

I think that anyone using pandoc to convert untrusted input would be wise to impose a timeout or other mechanism that kills processes taking too long (see the security notes in the manual).

[nachogarcia]

I've gone ahead and wrote a couple PRs based on this. I tried to keep them small, sorry if it resulted in a high number.
If you'd rather see the contributions in any other way, or you'd rather not spend time with XSLX now, let me know, is completely understandable.