Grammar index broken by #1858

[tkoeppe]

The changes from #1858 have caused a large number of index key errors in the grammar index, which show up as warnings in the index log file.

This is bad; we rely on clean logs to spot actual indexing errors. I was very close to adding an index log presubmit check, which would have blocked the application of the pull request.

Can you please fix this? (Perhaps by stripping out the formatting from the index key?)

[jensmaurer]

The warnings are like this:

## Warning (input = grammarindex.idx, line = 2206; output = grammarindex.ind, line = 8):
   -- Conflicting entries: multiple encaps for the same page under same key.

which is caused by index entries like this:

\indexentry{access-specifier@\textit  {access-specifier}|idxbfpage}{227}
\indexentry{access-specifier@\textit  {access-specifier}|hyperpage}{227}

which means the \grammarterm access-specifier was defined on page 227 and used on page 227. As you can see, the index keys are already the same. I have no idea how to fix this while keeping the (desired) boldness of defining mentions of \grammarterms.

[godbyk]

One solution would be to switch from makeindex to xindy. xindy can be instructed to give preference to the bold-faced page numbers. You can read some more details about this option here. In general, xindy is a more capable indexer.

[jensmaurer]

Editorial meeting consensus: Let's try xindy.

[godbyk]

I can write the xindy script to do this.

When a grammar term is both defined and used on the same page, do we want two page locators (one bold and one roman)? Or do we just want one (bold)?

[zygoloid]

Just one (in bold) seems preferable.

[godbyk]

@zygoloid Have a look at the grammar terms index with this configuration and see if it resolves the issue.

[godbyk]

I ran into a potential issue that's holding up my work on this PR. I've described it in detail elsewhere.

The short version is that xindy complains about a cross-reference that targets a top-level index entry when it doesn't have any page numbers itself (i.e., it only has page numbers for its subentries).

I'm still hoping to find a solution to that, but in the meantime we have a couple options:

1. Don't worry our pretty little heads about xindy errors. We can just treat them like warnings and investigate them when we have time and fix the true errors and ignore the rest.

2. Have cross-references always point to an index entry (or subentry) that has a page reference ("locator" in xindy parlance). This removes the errors reported by xindy, leaving us with only errors on our part (e.g., pointing cross-references at nonexistent entries). This would mean either directing cross-references to specific subentries when the main entry has no page references or adding a token page reference to the main entry to appease xindy.

Thoughts?

[tkoeppe]

Thank you so much for your work on this! Sorry this got delayed for so long, but I'm still very interested in a solution. Let's revisit this when we've sent the DIS, maybe we can work something out in time for the IS.

@zygoloid, @jensmaurer: just a note that we should keep an eye on this.