Counting words for Asian languages

[atimmer]

WordPress has build code to be able to count words in Asian languages. We should take this as inspiration. WordPress actually ships the word counting code only in the Chinese language pack, the third link shows where to find it.

Patch for word-count.js
https://core.trac.wordpress.org/ticket/20738
https://core.trac.wordpress.org/ticket/30966

Where to find the language pack:
https://core.trac.wordpress.org/ticket/33454

[monbauza]

Please inform the customer of conversation # 99154 when this conversation has been closed.

[monbauza]

Please inform the customer of conversation # 148759 when this conversation has been closed.

[paulovsky]

Any developments/ideas on this issue? Not being able to use Yoast with Chinese content is keeping the plugin away from 450 million internet users.

[monbauza]

Please inform the customer of conversation # 172656 when this conversation has been closed.

[a4jp-com]

Maybe a character count is better for Japanese/Chinese. One kanji character is usually takes up the space of 2 regular characters.

[terw-dan]

@a4jp-com thanks for your suggestion. So each kanji character can be seen as 1 word? Or is there more to it?

[paulovsky]

Japanese (also Chinese, Korean, Vietnamese) is a logographic language from the Han family; that means that not all characters represent morphemes: some morphemes are composed of more than one characters (see more here).

@terw-dan The approach is to take the word count as character count, mostly because each character counts as one "word" and there are no spaces between characters (at least in Chinese).

As for keyword analysis, if the user imputs the combination of two or more characters, that must be seen as one word.

[a4jp-com]

Individual kanji characters sometimes have a meaning but in some situations they are combined with a few other hiragana characters to make different words with different readings.

Kanji only:
下 (shita) down

Kanji with hiragana:
下さい (kudasai) please

Each character in either hiragana, katakana or kanji takes up the space of 2 english characters.

[terw-dan]

Thanks both for the explanation. This will come in helpful when we start implementing and testing this.

[a4jp-com]

Is there anything I can do to help?

[IreneStr]

@a4jp-com Thank you for your eagerness to help! At the moment, our main problem is the absence of spaces in Asian languages. Could you confirm that in Japanese there are no spaces between individual words as well.

[a4jp-com]

Sometimes a regular space is used but in other situations a Japanese space is used.

Here is the encoding for the unicode character 'IDEOGRAPHIC SPACE' (U+3000)

[IreneStr]

@a4jp-com Thank you for your reply. From googling some Japanese websites, I got the impression that spaces are generally speaking only used between sentences, but not between words.

In the following sentences, for example, there is a whitespace after 、 and 。.):

日本の文化を美術や音楽、演劇、映画からファッションやデザインまで幅広く世界に紹介しています。また、言葉を超えた共感の場をつくり出し、ともに創造する喜びをわかちあって、人と人との交流を深めていきます。

However, these white spaces are part of the 、(U+3001) and 。(U+3002) characters (so the space is not a separate character).

In what situations do people use the regular space or Japanese space?

[a4jp-com]

Regular spaces are used between the surname and given name.
山田 たろ
but sometimes no space is used or a Japanese space is used.
山田たろ
山田　たろ

It's kind of a design choice.

Regular spaces are also used when romaji or English phrases are used in advertising.

[idpokute]

Most of Japanese don't use space between words. Maybe Yoast can give the option; that turns off some deduction rules for Japanese.

[a4jp-com]

I've been living in Japan for 16 years and have worked as a system engineer in 3 Japanese companies. Spaces are used in sites here.

Especially in pages mixed with English words.

For example:
https://www.toshiba-newenergy.com/
OECD加盟国34ヵ国中33位
IEA Energy Balance of OECD Countries 2013

[IreneStr]

@idpokute @a4jp-com Thank you both for the information. It'll be very helpful when we want to implement this feature in the future.

[mmikhan]

Please inform the customer of conversation # 179998 when this conversation has been closed.

[a4jp-com]

Thank you very much @IreneStr.

[monbauza]

Please inform the customer of conversation # 184521 when this conversation has been closed.

[a4jp-com]

I have been following the rules set out in the plugin but I have lost 75% of my views on one Japanese site. I've gone from about 200 views a day down to about 50 views a day.

The only other change I made was changing the site to HTTPS. I thought that was meant to increase the ranking. Any ideas what could be causing the problem? https://agreatdream.com/word-lists/

Is this somehow linked to the count being off?

[ullivr]

i wrote my blog in chinese, really really really need this function.

[terw-dan]

@a4jp-com The wordcount is only shown as an indication. It is not something we (can) save to your post that has influence on your rankings. So it has to be something else that caused a decrease.

[a4jp-com]

I was just thinking as the numbers are wrong that when we make pages we might be adding titles that are too long etc

[saitaiky]

I write my blog in Chinese and Japanese. There is some example of my post title for you to test.
本願寺，錦市場，八坂神社 <==Chinese only
京阪神八日之旅 <==Chinese only
伏見稻荷大社，けんどん屋，奈良公園 <==Chinese mix Japanese

Please help to fix it , we do need this amazing function! it keeps telling us our posts are poor makes us sad.
Thank you so so much

[mmikhan]

Please inform the customer of conversation # 192085 when this conversation has been closed.

[saitaiky]

has it been fixed yet? guys

[a4jp-com]

I'd love to find out the character count through this plugin.

[mmikhan]

Please inform the customer of conversation # 198257 when this conversation has been closed.

[a4jp-com]

Any ideas on checking the word count?

[a4jp-com]

Is anyone still working on this?

[amboutwe]

Please inform the customer of conversation # 516867 when this conversation has been closed.

[amboutwe]

Please inform the customer of conversation # 538393 when this conversation has been closed.

[hawm]

Still a problem at Sep 2019.

[mayada-ibrahim]

Please inform the customer of conversation # 565766 when this conversation has been closed.

[mmikhan]

Please inform the customer of conversation # 585879 when this conversation has been closed.

[a4jp-com]

Is anyone working on this anymore? This has been here for about 4 years.

[Djennez]

This is currently not being worked on, it is a feature request that might get implemented in the future.

[a4jp-com]

Okay. Thanks for the honest reply @Djennez. I'll post this update in the plugin download area of WordPress. No one has said this up till now which isn't very good. This should have been made clear years ago.

Is it okay to fork the current plugin and make a Japanese version. I'll just edit the character count code. What is the licence of the current free version? Is it a General Public License?

[Pcosta88]

Please inform the customer of conversation # 598441 when this conversation has been closed.

[7creo]

Okay. Thanks for the honest reply @Djennez. I'll post this update in the plugin download area of WordPress. No one has said this up till now which isn't very good. This should have been made clear years ago.

Is it okay to fork the current plugin and make a Japanese version. I'll just edit the character count code. What is the licence of the current free version? Is it a General Public License?

Did you success on this yet? I still don't understand how this situation can be ignored by the developers for such a long time. I would never purchase a paid version of that with this issue, as it makes the plugin 50% useless.

[Pcosta88]

Please inform the customer of conversation # 607816 when this conversation has been closed.

[michaelbriantina]

https://wordpress.org/support/topic/cjk-word-count/

[a4jp-com]

Can you include a conditional statement based on the page language that either counts words or characters based on the language set?

Example 1:

var lang = document.documentElement.lang if (lang === ‘ja’) { ...do something - Japanese character count code here... }

Example 2:

 <?php if(ICL_LANGUAGE_CODE=='zh' || ICL_LANGUAGE_CODE=='ja' || ICL_LANGUAGE_CODE=='ko'){ ?>
...Chinese/Japanese/Korean character count code...
<?php } ?> 

I'm not sure how you have separated other languages but I'm sure you already have code like this in your plugin for other languages.

[crpeng]

I also need this improvement for Chinese language.

[a4jp-com]

T-T. Whatever happened to the programmer that was assigned to fixing this problem?

[suascat]

Please inform the customer of conversation # 649514 when this conversation has been closed.

[priscillamc]

https://wordpress.org/support/topic/seo-in-japanese/

[michaelbriantina]

Please inform the customer of conversation # 729318 when this conversation has been closed.

[a4jp-com]

Is there a way to just get a character count instead of word count? You already have the code in the plugin. You just need a function for the option.

[ogodoabiola]

Please inform the customer of conversation #744721 when this conversation has been closed.

[Isildur00]

Please inform the customer of conversation #760565 when this conversation has been closed.

[a4jp-com]

WordPress has build code to be able to count words in Asian languages. We should take this as inspiration. WordPress actually ships the word counting code only in the Chinese language pack, the third link shows where to find it.

Patch for word-count.js
https://core.trac.wordpress.org/ticket/20738
https://core.trac.wordpress.org/ticket/30966

Where to find the language pack:
https://core.trac.wordpress.org/ticket/33454

Can we find out what is happening if you already have the code please?

[amyuki]

Can you just simply counting words like what WordPress Editor do?

[michaelbriantina]

Please inform the customer of conversation # 894142 when this conversation has been closed.

[mmikhan]

We are going to close the issue since supporting for Asian language is a generic term as there are a whole lot languages available in Asia. Therefore, we will split the issue and open new ones for specific languages. If you want to see if Yoast SEO has support for Chinese, Korean, or Vietnamese, do feel free to open a specific issue (if there's none already) so that we can see how many users are requesting for what language.

That said, Yoast SEO v18.0 has already included support for Japanese (which is one of the Asian languages as well.)

[a4jp-com]

Thanks for adding Japanese. I never noticed you did that. ♥