Clicking broken Help entries results in Help closing altogether

[Wedge009]

Game and System Information

master and 1.14 on Windows

Describe the bug

I found that when I made mistakes while working on resolving #3064, broken Help entries would simply 'crash' the Help dialogue when clicked. But interestingly, I first discovered this while clicking certain pages under French translation, even though the same page works with English and even German. I did not notice any error messages in the logs resulting from this.

To reproduce

1. Switch to French language
2. Open Help
3. Go to Terrains (root page) and observe Help closing
4. Switch to English or German language
5. Go to Terrains again and note the page works as normal

Expected behaviour

French Terrains page should display as normal, even if not all text is translated.

[stevecotton]

Thanks for logging this - it was noticed in #4359 but we forgot about it again.

[Wedge009]

Oh, I didn't realise this was the cause. Where do these carets appear to cause the page to break? Is it just the terrain code? I wonder why English and German Terrain pages work.

[stevecotton]

The issue is malformed text - the caret handling truncated the text. With the log message from PR #4514 it's a lot easier to work out what's going wrong, although I think the current issue should stay open until the help is fixed to do a best-effort when handling malformed text.

The French terrains page is having trouble with the ' in d'eau:

wesnoth/po/wesnoth-help/fr.po

Lines 2984 to 2995 in bdc26a1

	msgstr ""
	"\n"
	"\n"
	"Une exception notable concerne les ponts, comme les <italic>text='ponts au-"
	"dessus d'eau peu profonde'</italic>, <italic>text='gués'</italic> et "
	"<italic>text='ponts au-dessus de crevasses'</italic>. Les gués sont "
	"facilement franchissables par des ondins ou des humains — toutes les unités "
	"se déplaçant sur un gué profitent de la meilleure défense et du meilleur "
	"mouvement entre la plaine et l'eau peu profonde, au lieu du pire des deux. "
	"De façon similaire, les ponts au-dessus des crevasses sont franchissables "
	"par les unités qui ne volent pas (sans surprise).\n"
	"\n"

[Wedge009]

Ayoh, c'est terrible!

Merci.

[Wedge009]

Actually, thinking about it, this wouldn't be a problem if a proper apostrophe or 'right quote' is used there - is this correct? Because the straight quote is used for the text formatting. I'm sure French isn't the only language that might have this issue... wondering if this is another thing we need to fix and announce in #4436.

[jostephd]

Actually, thinking about it, this wouldn't be a problem if a proper apostrophe or 'right quote' is used there - is this correct?

There are several aspects to this.

• As you say, <format>text='foo’bar'</format> (with a Unicode quote in there) would parse correctly.

• We should not forbid translations to use ASCII single quotes in there. That is the convention the English text uses, but we shouldn't impose that convention on translations. As far as the help system is concerned, there should be an escaping syntax that makes it possible to use escaped single quotes in the values of text attributes. (The serialization function I mentioned in Add Time of Day Bonuses to Help (master) #4500 (comment) would then generate such syntax when it serializes the config object [format] text="foo'bar" [/format].)

• The French translation currently has a syntax error. That needs to be fixed.

• Such errors could be caught by Travis, in principle.

• The help dialog shouldn't just close when there's a markup error. It should show an error message, and perhaps also show the user the raw description string, like Pango does when there's a Pango markup syntax error. (This would be redundant if we convert the help system to Pango markup, of course, but if it's not much effort to implement, we might do it now as a stopgap until the conversion.)

[CelticMinstrel]

The easiest quick fix would probably be to replace a sequence of two single quotes with a single apostrophe, same as how double quotes are escaped in full WML syntax. (The help markup is technically an alternative WML syntax.)

But I wonder if it would be better to convert it to a more Pango-like markup? Use the Pango tags like <i>, <b> etc where possible, replace <format> with a subset of the <span> tag, and reformat the other tags to be Pango-like, eg <ref dst='topic-name' force='false'>text</ref>, <img align='left' float='yes'>image/path.png</img>. Not sure how <jump> would look though, I'm assuming Pango doesn't have empty or self-closing tags like HTML or XML...

I guess that does mean writing a parser for the Pango markup though (unless Pango exposes its parser in a way that can be used independent of the Pango layout system, but even then there's the extra extension tags to deal with).