ToAscii: Some russian texts in Cyrillic (Azbuka) are wrong rewrited #203

Activity

1. dg commented on Oct 29, 2019

   

   dg

   on Oct 29, 2019

   Last edited by dg

   Member

   More actions

   Do you have enabled Transliterator?

   utils/src/Utils/Strings.php

   Line 141 in c133e18

   if ($transliterator === null && class_exists('Transliterator', false)) {

   And what implementation of iconv are you using?

   utils/src/Utils/Strings.php

   Line 154 in c133e18

   if (ICONV_IMPL === 'glibc') {
2. janbarasek commented on Oct 29, 2019

   

   janbarasek

   on Oct 29, 2019

   ContributorAuthor

   More actions

   @dg Yes, Transliterator was instanced and ICONV_IMPL is libiconv.

   I detected it by:

   /**
    * Converts UTF-8 string to ASCII.
    */
   public static function toAscii(string $s): string
   {
   	static $transliterator = null;
   	if ($transliterator === null && class_exists('Transliterator', false)) {
   		$transliterator = \Transliterator::create('Any-Latin; Latin-ASCII');
   	}
   	dump(['transliterator' => $transliterator, 'iconv' => ICONV_IMPL]);

   Dumped:

   

   Thanks.
3. dg commented on Oct 29, 2019

   

   dg

   on Oct 29, 2019

   Member

   More actions

   Transliterator converts я to â (according to ISO 9 https://cs.wikipedia.org/wiki/ISO_9) and then â to a.

   Probably solution is to add strtr for chars Я я Ю ю and maybe some others.
4. janbarasek commented on Oct 29, 2019

   

   janbarasek

   on Oct 29, 2019

   ContributorAuthor

   More actions

   I think we can use (for Czech language):

   [
   	'ё' => 'jo',
   	'ъ' => '',
   	'ы' => 'y',
    	'ь' => '', 
   	'э' => 'eh', 
   	'ю' => 'ju',
   	'я' => 'ja',
   ]

   But for correct behavior is very important use target language (which can be loaded from second parameter string $language = 'en').

   And table for English:

   [
   	'ё' => 'jo',
   	'ъ' => '',
   	'ы' => 'y',
    	'ь' => '', 
   	'э' => 'e', 
   	'ю' => 'yu',
   	'я' => 'ya',
   ]
5. dg

   closed this as completedin 5716775on Jan 3, 2020
6. dg

   added 2 commits that reference this issue on Jan 3, 2020

   Strings::toAscii transliterates я/ю as ya/yu [Closes #203]

   This issue will close once commit 88187c7 is merged into the 'master' branch.

   88187c7

   Strings::toAscii transliterates я/ю as ya/yu [Closes #203]

   This issue will close once commit d6cd63d is merged into the 'master' branch.

   d6cd63d