Smarty does a poor job at German apostrophes

[gandaro]

Configuring it with the correct values for 'left-single-quote' and 'right-single-quote', it gives wrong apostrophes, for example:

"O'Reilly" → "O‘Reilly" (correct is "O’Reilly")
"Alice'" → "Alice‘" (correct is "Alice’")

{
    'markdown.extensions.smarty': {
        'substitutions': {
            'left-single-quote': '‚',
            'right-single-quote': '‘',
            'left-double-quote': '„',
            'right-double-quote': '“'
        }
    }
}

So you should be able to tell smarty: I want apostrophes instead of right-single-quotes.

[waylan]

Thank you for your report.

Unfortunately, I am unsure of the exact problem you are reporting. Is this connected to #397 or is that separate? It occurs to me that if your were passing in the configs incorrectly due to the documentation error, that that was causing the wrong type of quote to be used. If so, then this may not actually be an issue and can be closed.

If, on the other hand, this is an actual separate issue, all I can say is that I can't possibly factor in every edge case for every language out there. It is not clear to me if you need to set a different substitution in the configs or if the logic which selects the type of quote needs to be improved for your specific needs. If it is the later, my inclination is to suggest that the extension should be forked and maintained as a language specific fork that meets all of the nuances of that specific language.

In fact, the existing support for substitutions was provided by a third party contributor who also provided the (apparently broken) documentation. That person informed me that the example in the docs worked for German. Personally, I have no knowledge of German or what requirements that language has. If the issue is that we need better docs, then I am happy to accept patches (although I don't what to document German (or any language's) syntax rules). However, as stated above, anything except very minor code changes (which benefit users of all languages) are best addressed with a fork maintained by someone who understands the specific needs of that fork.

[waylan]

Taking another look at your report, it would appear that you require apostrophes and right single quotes to be two separate and different characters. In English, those two uses generally call for the same character. As there is no difference in practice, the code does not differentiate between them. Presumably, a whole additional feature would need to be added to the extension to support apostrophes. I suppose the English default could be for "apostrophes" and "right-single-quotes" to have the same character assigned to them and offer the option to provide independent substitutions for each for other languages. That would certainly be an addition that could potentially support multiple languages.

That said, I have neither the time, the need, nor the interest in working on this myself. Of course, patches are welcome.

[mitya57]

I think we just need to make closingSingleQuotesRegex2 produce an apostrophe, not a single quote.

However this way not all language problems will be solved, i.e. that regex contains a case for English-specific 's ending.

[mitya57]

After looking at it again, it is not that simple — that regex matches quotes on word endings and the cases like Paul's.

I know a bit of German, but I am not an expert — are all use cases for apostrophe limited by 's, or are there other ones? If there are no other cases, then I will just split that regex into two regexes and make one return an apostrophe instead of closing-single-quote.

[waylan]

I think that is a reasonable approach to take if we are going to change anything. I would suggest that the "apostrophe" regex cover the uses discussed by Wikipedia. Don't forget about non-English Proper Names, which are buried deep in that page. I would think that would mostly be pretty straightforward (eg: "\w'\w"). However, a couple issues may arise with plural years (eg: the 1990's or the '90's) and nouns ending in an "s" (or "s" or "z" sound) where a second "s" is not added after the the apostrophe (whether it is plural or not).

The non-plural ending-in-s is a rare case. But even the common plural case can be tricky. Consider these examples:

1. "He stated that they were his brother's tools." (the tools belong to one brother)
2. "He stated that they were his 'brother's' tools." (same as 1 except that 'brother' is in quotes because it is a title -- perhaps not his 'real brother')
3. "He stated that they were his brothers' tools." (the tools belong to multiple brothers -- this is an apostrophe not a right single quote)
4. "He stated that they were his 'brothers'' tools." (same as 2 with multiple 'brothers' (as a quoted title) - this is an apostrophe followed by a right single quote)

It is not clear to me how to consistently determine which is an apostrophe and which is a right quote in every one of those cases with a regex. If it is not reasonably possible, then I say we should not bother making any change.

[gandaro]

I know a bit of German, but I am not an expert — are all use cases for apostrophe limited by 's, or are there other ones?

German use cases:

• Alice’ Abenteuer im Wunderland
• Max’ und Moritz’ Scherze
• Johannes’ Aufgabe
• Dern’sches Gelände
• Andrea’s Blabla (actually a “greengrocer’s apostrophe”)
• Wie geht’s?
• Ku’damm
• ’ne Kippe
• Kommen S’ nur herein!
• mit’m Fahrrad fahr’n

Might not cover all of them.

[mitya57]

@gandaro Thanks! In half of your examples the apostrophe is at the beginning (or end) of the word, so it will be impossible to distinguish it from opening/closing quote...

[waylan]

That's what I expected. Therefore I am closing this as wontfix.