Keywords count bugged in page analysis tab #703

Closed
deinezauber opened this Issue Mar 1, 2014 · 25 comments

Comments

Projects
None yet
4 participants
@deinezauber

Environment: Wordpress 3.7.1, Wordpress SEO 1.4.25

Description: after the plugin upgrade, the keyword count is always zero in the "Page analysis" tab of the Wordpress SEO frame inside the update post page, even if the keywords are copied and pasted multiple times inside the post content (and after publishing, of course).

Steps to repro:

  1. create a new post with the keywords in the title, the meta description and the post content;
  2. save the draft.

Expected behaviour: the tab "Page analysis" shows the correct number of times that the keywords appear in the post content.

Observed behaviour: the tab "Page analysis" shows always a zero count for keywords. The general tab shows correct data instead.

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 1, 2014

Contributor

I've tested this in WP3.7 and WP3.8.1 with WPSEO 1.4.25 and cannot reproduce the issue.

Can you give an example text + keyword with which you are experiencing this issue ?

Contributor

jrfnl commented Mar 1, 2014

I've tested this in WP3.7 and WP3.8.1 with WPSEO 1.4.25 and cannot reproduce the issue.

Can you give an example text + keyword with which you are experiencing this issue ?

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 2, 2014

Thank you for your answer! Sure, I've created a new post from scratch and set this fields (sorry but it's all in Italian and Japanese language):

Post title: I numeri in giapponese - test

Wordpress SEO Keywords: numeri in giapponese

Post text: I numeri in giapponese
I numeri in giapponese - Note
Fra i numeri in giapponese da 0 a 10 troviamo diversi casi di numeri con più di una pronuncia, come 0, 4, 7 e 9; la pronuncia va scelta a seconda dei numeri che vengono prima o dopo. Nel caso particolare del numero 4, la pronuncia よ viene usata solamente per leggere i minuti di un orario.
I numeri da 11 a 19 sono la combinazione del 10 seguito dai numeri da 1 a 9; bisogna osservare che le pronunce di 14, 17 e 19 usano le doppie letture di 4, 7 e 9.
Le decine sono ottenute dai numeri da 2 a 9 seguito dal numero 10; in questo caso per 40, 70 e 90 viene scelta solo una delle due letture disponibili dei numeri 4, 7 e 9.
Le centinaia seguono lo stesso schema visto per le decine e si noti la differente pronuncia di 300, 600 e 800; per le migliaia bisogna invece porre l'attenzione sui numeri 3 000 e 8 000.
Diversamente dalla numerazione occidentale, che legge i numeri a gruppi di tre cifre (evidenziando cioè le migliaia, i milioni e i miliardi), nel sistema numerico giapponese si leggono i numeri a gruppi di quattro cifre; per esempio, il numero 11111111 si può leggere:

  • in italiano, undici milioni centoundicimila centoundici,
  • in giapponese, せんひゃくじゅういちまん せんひゃくじゅういち.
    Cento milioni corrispondono ad un おく, ovvero 10 000まん = いちおく (cfr. Sistema di numerazione in Wikipedia)

Screenshots: the first screenshot shows that the keyword count on the post fields is correctly performed.

screenshot-1

The second screenshot shows instead the problem with the keyword count indicator (the third one from the top)

screenshot-2

Thank you for your answer! Sure, I've created a new post from scratch and set this fields (sorry but it's all in Italian and Japanese language):

Post title: I numeri in giapponese - test

Wordpress SEO Keywords: numeri in giapponese

Post text: I numeri in giapponese
I numeri in giapponese - Note
Fra i numeri in giapponese da 0 a 10 troviamo diversi casi di numeri con più di una pronuncia, come 0, 4, 7 e 9; la pronuncia va scelta a seconda dei numeri che vengono prima o dopo. Nel caso particolare del numero 4, la pronuncia よ viene usata solamente per leggere i minuti di un orario.
I numeri da 11 a 19 sono la combinazione del 10 seguito dai numeri da 1 a 9; bisogna osservare che le pronunce di 14, 17 e 19 usano le doppie letture di 4, 7 e 9.
Le decine sono ottenute dai numeri da 2 a 9 seguito dal numero 10; in questo caso per 40, 70 e 90 viene scelta solo una delle due letture disponibili dei numeri 4, 7 e 9.
Le centinaia seguono lo stesso schema visto per le decine e si noti la differente pronuncia di 300, 600 e 800; per le migliaia bisogna invece porre l'attenzione sui numeri 3 000 e 8 000.
Diversamente dalla numerazione occidentale, che legge i numeri a gruppi di tre cifre (evidenziando cioè le migliaia, i milioni e i miliardi), nel sistema numerico giapponese si leggono i numeri a gruppi di quattro cifre; per esempio, il numero 11111111 si può leggere:

  • in italiano, undici milioni centoundicimila centoundici,
  • in giapponese, せんひゃくじゅういちまん せんひゃくじゅういち.
    Cento milioni corrispondono ad un おく, ovvero 10 000まん = いちおく (cfr. Sistema di numerazione in Wikipedia)

Screenshots: the first screenshot shows that the keyword count on the post fields is correctly performed.

screenshot-1

The second screenshot shows instead the problem with the keyword count indicator (the third one from the top)

screenshot-2

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 2, 2014

Contributor

@deinezauber I've tested your text in WPSEO 1.5-beta with WP 3.8.1 and find no problems. I don't believe we changed anything particular about the keyword density calculation in v1.5, but it does look solved to me there.

Would you be willing to test this yourself ? You can download the v1.5 branch here in GitHub. Please do make a backup of the database before testing and don't test in a production/live environment.

screenshot

Contributor

jrfnl commented Mar 2, 2014

@deinezauber I've tested your text in WPSEO 1.5-beta with WP 3.8.1 and find no problems. I don't believe we changed anything particular about the keyword density calculation in v1.5, but it does look solved to me there.

Would you be willing to test this yourself ? You can download the v1.5 branch here in GitHub. Please do make a backup of the database before testing and don't test in a production/live environment.

screenshot

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 2, 2014

Thank you for your kind support, then tomorrow I'll have a try on a test environment running Wordpress 3.8.1 and Wordpress SEO 1.4.25 (I don't expect a different behaviour but I don't want to mix too many variables at once); later on I'll update Wordpress SEO to 1.5 and come back here with the results. By the way, when are you planning to release this latest version?
Thank you again,
Nicola

Thank you for your kind support, then tomorrow I'll have a try on a test environment running Wordpress 3.8.1 and Wordpress SEO 1.4.25 (I don't expect a different behaviour but I don't want to mix too many variables at once); later on I'll update Wordpress SEO to 1.5 and come back here with the results. By the way, when are you planning to release this latest version?
Thank you again,
Nicola

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 3, 2014

Contributor

Ok, I look forward to hearing your results.

By the way, when are you planning to release this latest version?

As with most open source: when It's ready ;-) or in this case, when we're satisfied that testing has caught most or at least, the worst bugs.

Contributor

jrfnl commented Mar 3, 2014

Ok, I look forward to hearing your results.

By the way, when are you planning to release this latest version?

As with most open source: when It's ready ;-) or in this case, when we're satisfied that testing has caught most or at least, the worst bugs.

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 3, 2014

Ok, I've tried the 1.4.25 Wordpress SEO on a 3.8.1 Wordpress and it's working...

I don't know what to think because recently I've updated only the Wordpress SEO on my production environment (WP 3.7.1) without any other update or modification and it's not working, as I've shown you with the screenshots.
I don't think there's something wrong with my environment since many people on Wordpress SEO support forum are notifying the same problems of mine.

Anyway, I have already planned an update of my production environment to the WP 3.8.1 so I hope that everything will work fine again.
I'll report the results of the update, so please don't close this issue until my last comment.
Thank you,
Nicola

Ok, I've tried the 1.4.25 Wordpress SEO on a 3.8.1 Wordpress and it's working...

I don't know what to think because recently I've updated only the Wordpress SEO on my production environment (WP 3.7.1) without any other update or modification and it's not working, as I've shown you with the screenshots.
I don't think there's something wrong with my environment since many people on Wordpress SEO support forum are notifying the same problems of mine.

Anyway, I have already planned an update of my production environment to the WP 3.8.1 so I hope that everything will work fine again.
I'll report the results of the update, so please don't close this issue until my last comment.
Thank you,
Nicola

@Sandmanyz

This comment has been minimized.

Show comment Hide comment
@Sandmanyz

Sandmanyz Mar 3, 2014

I have the same problem. Yesterday i installed the new version 1.4.25 and keyword count is 0%. Today i updated to wordpress 3.8.1 and it still 0%.

I have the same problem. Yesterday i installed the new version 1.4.25 and keyword count is 0%. Today i updated to wordpress 3.8.1 and it still 0%.

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 3, 2014

Contributor

@Sandmanyz same problem = same answer: so far I have not been able to reproduce the problem. Unless you provide me with more information or a text + keyword which will allow me to reproduce the issue, there is nothing we can do.

Contributor

jrfnl commented Mar 3, 2014

@Sandmanyz same problem = same answer: so far I have not been able to reproduce the problem. Unless you provide me with more information or a text + keyword which will allow me to reproduce the issue, there is nothing we can do.

@iantvc15

This comment has been minimized.

Show comment Hide comment
@iantvc15

iantvc15 Mar 3, 2014

Hi, I have the same problems on every wordpress site I have. JRFNL no need to reproduce the problem, you can log into one of my websites to have a look. Let me know if you'd like to do so.

iantvc15 commented Mar 3, 2014

Hi, I have the same problems on every wordpress site I have. JRFNL no need to reproduce the problem, you can log into one of my websites to have a look. Let me know if you'd like to do so.

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 3, 2014

Contributor

@iantvc15 LOL... let me remind you.. this is a free plugin... without any free official support... are you willing to pay me to debug your website ? If not, don't you think you are asking a bit much ?

Contributor

jrfnl commented Mar 3, 2014

@iantvc15 LOL... let me remind you.. this is a free plugin... without any free official support... are you willing to pay me to debug your website ? If not, don't you think you are asking a bit much ?

@iantvc15

This comment has been minimized.

Show comment Hide comment
@iantvc15

iantvc15 Mar 3, 2014

I am confused. If this place is where people identify issues and then solve them then I would think people may want to see a problem in its environment. I don't expect you to do any work for me (read my post again). I thought I was being helpful when offering you direct access to see this problem. Clearly your post a few hours ago shows you are trying to reproduce the problem - now I am offering you access to it already created. And I get slated for it - typical knee jerk reaction with little thought about the real objective. This will be my first and last experience here!

iantvc15 commented Mar 3, 2014

I am confused. If this place is where people identify issues and then solve them then I would think people may want to see a problem in its environment. I don't expect you to do any work for me (read my post again). I thought I was being helpful when offering you direct access to see this problem. Clearly your post a few hours ago shows you are trying to reproduce the problem - now I am offering you access to it already created. And I get slated for it - typical knee jerk reaction with little thought about the real objective. This will be my first and last experience here!

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 3, 2014

Contributor

@iantvc15 Ok, let's make this very simple: if an issue can not be reproduced on a standard clean WP install, it is more likely than not a problem in your set-up, not with the plugin. So no, I do not want to see the problem in it's environment as most likely that environment is the problem.
I'll repeat: good input to reproduce the issue is welcome. A request to debug your website for free is not.

Contributor

jrfnl commented Mar 3, 2014

@iantvc15 Ok, let's make this very simple: if an issue can not be reproduced on a standard clean WP install, it is more likely than not a problem in your set-up, not with the plugin. So no, I do not want to see the problem in it's environment as most likely that environment is the problem.
I'll repeat: good input to reproduce the issue is welcome. A request to debug your website for free is not.

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 3, 2014

@jrfnl : I've just updated my WP to 3.8.1 and the keywords count is still at zero; now my production and test environment has for sure the same code running on them but results are different, so there's something related to environments that I can't figure it out yet...
I've had a look to the source code of the plugin - especially the algorithm behind the indicator computation - but I didn't get any hint for understanding what's wrong with my environment.
I'd like to ask you if there is some settings for a detailed logging or something like that for helping me with environment analysis.
Thank you,
Nicola

@jrfnl : I've just updated my WP to 3.8.1 and the keywords count is still at zero; now my production and test environment has for sure the same code running on them but results are different, so there's something related to environments that I can't figure it out yet...
I've had a look to the source code of the plugin - especially the algorithm behind the indicator computation - but I didn't get any hint for understanding what's wrong with my environment.
I'd like to ask you if there is some settings for a detailed logging or something like that for helping me with environment analysis.
Thank you,
Nicola

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 4, 2014

Contributor

@deinezauber Normally I'd advise a stack trace, but there are no PHP errors in this code, though there may be some in/caused by other plugins.

First thing to try is to open the post for editing and save as draft/save changes (if published) to see if that makes any difference.

If it doesn't, I'd suggest de-activating all other plugins and switching to the standard WP theme to see if that makes a difference. If it does, activate the other plugins/switch the theme one by one to see when the problem start occurring.

Contributor

jrfnl commented Mar 4, 2014

@deinezauber Normally I'd advise a stack trace, but there are no PHP errors in this code, though there may be some in/caused by other plugins.

First thing to try is to open the post for editing and save as draft/save changes (if published) to see if that makes any difference.

If it doesn't, I'd suggest de-activating all other plugins and switching to the standard WP theme to see if that makes a difference. If it does, activate the other plugins/switch the theme one by one to see when the problem start occurring.

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 4, 2014

Contributor

Based on the sample texts I received here and in some related issues, I have made some small improvements to the keyword density calculation for non-latin, non-ideograph based languages by means of commit 97d0973.
Hopefully that will yield more consistent results. The changes are included in the the v1.5 branch. You will need to re-download the branch if you want to test the changes.

Related issues: #703, #681, #349, #264 and #145.

Contributor

jrfnl commented Mar 4, 2014

Based on the sample texts I received here and in some related issues, I have made some small improvements to the keyword density calculation for non-latin, non-ideograph based languages by means of commit 97d0973.
Hopefully that will yield more consistent results. The changes are included in the the v1.5 branch. You will need to re-download the branch if you want to test the changes.

Related issues: #703, #681, #349, #264 and #145.

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 4, 2014

@jrfnl Hi Juliette, thank you for your precious advices!
Unfortunately I haven't had any luck... I've followed this schema: deactivated one by one all of my plugins and also switched the theme to twenty fourteen: after any change I've updated the page (which data I've sent to you) and checked if the "Page analysis" was changed.
There's something weird in my production environment and I'll go for a debug with some echos and print_r in order to understand better where's the hitch.
I'll come back with some results within this week.
Have a nice day, ciao!
Nicola

@jrfnl Hi Juliette, thank you for your precious advices!
Unfortunately I haven't had any luck... I've followed this schema: deactivated one by one all of my plugins and also switched the theme to twenty fourteen: after any change I've updated the page (which data I've sent to you) and checked if the "Page analysis" was changed.
There's something weird in my production environment and I'll go for a debug with some echos and print_r in order to understand better where's the hitch.
I'll come back with some results within this week.
Have a nice day, ciao!
Nicola

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 4, 2014

@jrfnl I've compared PROD and TEST environments adding some echos after the line no.1779 of the file "admin/class-metabox.php".
After updating the same post content, the resulting echos are different especially about the variable $res because in the PROD environment is equal to zero and in TEST is equal to 2; in order to go on with my analysis I need to undestand why there is this crucial difference: Juliette, could you please give me a hint?
Thank you,
Nicola

@jrfnl I've compared PROD and TEST environments adding some echos after the line no.1779 of the file "admin/class-metabox.php".
After updating the same post content, the resulting echos are different especially about the variable $res because in the PROD environment is equal to zero and in TEST is equal to 2; in order to go on with my analysis I need to undestand why there is this crucial difference: Juliette, could you please give me a hint?
Thank you,
Nicola

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 4, 2014

@jrfnl Sorry Juliette, silly question... that $res variable is passed by reference to the function preg_match_all that popoulate it with an array of matches; so the question becomes: why the preg_match_all function does not match anything in PROD whereas in TEST there are 2 matches? I'll investigate on this now!

@jrfnl Sorry Juliette, silly question... that $res variable is passed by reference to the function preg_match_all that popoulate it with an array of matches; so the question becomes: why the preg_match_all function does not match anything in PROD whereas in TEST there are 2 matches? I'll investigate on this now!

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 4, 2014

Contributor

@deinezauber First things first, which WP and WPSEO versions are you using now in test ? and which in production ?

Next, if the $res variable results are different, my first suggestion would be to test both in your test environment as well as your production environment whether PCRE has been compiled with utf-8 and unicode support. You can test this using the below code snippet.

If either of these two tests fail, you've probably found the cause of your problem. Though to be fair, you should have problems with more plugins and possibly even WP core if PCRE is compiled without this. You will need to ask your webhost to fix this.

print 'Testing PCRE for utf-8 support <br/>';
if ( ! @preg_match( '`^.$`u', 'ñ') ) {
    print 'PCRE has not been compiled with UTF-8 support.<br />';
}
else if ( ! @preg_match( '`^\pL$`u', 'ñ' ) ) {
    print 'PCRE has not been compiled with Unicode property support.<br />';
}
Contributor

jrfnl commented Mar 4, 2014

@deinezauber First things first, which WP and WPSEO versions are you using now in test ? and which in production ?

Next, if the $res variable results are different, my first suggestion would be to test both in your test environment as well as your production environment whether PCRE has been compiled with utf-8 and unicode support. You can test this using the below code snippet.

If either of these two tests fail, you've probably found the cause of your problem. Though to be fair, you should have problems with more plugins and possibly even WP core if PCRE is compiled without this. You will need to ask your webhost to fix this.

print 'Testing PCRE for utf-8 support <br/>';
if ( ! @preg_match( '`^.$`u', 'ñ') ) {
    print 'PCRE has not been compiled with UTF-8 support.<br />';
}
else if ( ! @preg_match( '`^\pL$`u', 'ñ' ) ) {
    print 'PCRE has not been compiled with Unicode property support.<br />';
}
@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 4, 2014

Contributor

@iantvc15 You may want to take note of my previous comment and run those tests on your website too.

Contributor

jrfnl commented Mar 4, 2014

@iantvc15 You may want to take note of my previous comment and run those tests on your website too.

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 4, 2014

Contributor

@jdevalk @tacoverdo It might be an idea to add this test for PCRE unicode support to the FAQ on the Yoast website as part of an answer for issues with text analysis ? Additional note to add to the FAQ answer: even with PCRE unicode support, text analysis (or at least the keyword density part) will not work for ideograph based languages (Japanese, Chinese etc)

Contributor

jrfnl commented Mar 4, 2014

@jdevalk @tacoverdo It might be an idea to add this test for PCRE unicode support to the FAQ on the Yoast website as part of an answer for issues with text analysis ? Additional note to add to the FAQ answer: even with PCRE unicode support, text analysis (or at least the keyword density part) will not work for ideograph based languages (Japanese, Chinese etc)

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 4, 2014

@jrfnl Thank you very much Juliette!!!

Both environments are running a WP 3.8.1 and Wordpress SEO 1.4.25.

Now, I've found one more (important) detail about the two environments: PROD is based on a PHP Version 5.3.3-7+squeeze14 whereas the TEST environment is based on a PHP Version 5.4.4-14+deb7u3.
Adding also the snippet code you gave me to my echoing section, I've also detected that PCRE has not been compiled with UTF-8 support.

So now I'll ask to my Hosting Provider an update or a server migration, in order to have an updated deployment; I think that my question has been answered so thank you for your kind support and please feel free to ask me more information, if you need to improve Wordpress SEO documentation or other stuff.

Thank you again,
Nicola

@jrfnl Thank you very much Juliette!!!

Both environments are running a WP 3.8.1 and Wordpress SEO 1.4.25.

Now, I've found one more (important) detail about the two environments: PROD is based on a PHP Version 5.3.3-7+squeeze14 whereas the TEST environment is based on a PHP Version 5.4.4-14+deb7u3.
Adding also the snippet code you gave me to my echoing section, I've also detected that PCRE has not been compiled with UTF-8 support.

So now I'll ask to my Hosting Provider an update or a server migration, in order to have an updated deployment; I think that my question has been answered so thank you for your kind support and please feel free to ask me more information, if you need to improve Wordpress SEO documentation or other stuff.

Thank you again,
Nicola

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 4, 2014

Contributor

@deinezauber Hi Nicola, I'm glad this helped and hope your webhost will upgrade you soon.
Your offer to help is definitely welcome. If there's anything you see that could do with improving, please feel free to send in a pull request or open an issue with - for instance - an updated documentation text for the website. You can also contribute to the translations if you like. ;-) /cc @jdevalk

Contributor

jrfnl commented Mar 4, 2014

@deinezauber Hi Nicola, I'm glad this helped and hope your webhost will upgrade you soon.
Your offer to help is definitely welcome. If there's anything you see that could do with improving, please feel free to send in a pull request or open an issue with - for instance - an updated documentation text for the website. You can also contribute to the translations if you like. ;-) /cc @jdevalk

@jrfnl jrfnl closed this Mar 4, 2014

@deinezauber

This comment has been minimized.

Show comment Hide comment
@deinezauber

deinezauber Mar 5, 2014

@jrfnl Hi Juliette, maybe I can help with the translations from English to Italian; I don't have so much spare time but if the philosophy is "It's ready when it's ready" then it can fit with my daily tasks.
Please send me a private email with the details of how contributors work for Yoast.
Thank you,
Nicola

@jrfnl Hi Juliette, maybe I can help with the translations from English to Italian; I don't have so much spare time but if the philosophy is "It's ready when it's ready" then it can fit with my daily tasks.
Please send me a private email with the details of how contributors work for Yoast.
Thank you,
Nicola

@jrfnl

This comment has been minimized.

Show comment Hide comment
@jrfnl

jrfnl Mar 5, 2014

Contributor

@deinezauber Hi Nicola, if you go to http://translate.yoast.com/projects/wordpress-seo, you can register and contribute to the translation. The current Italian translation is incomplete and some of the completed phrase might need checking, so it'd be great if you contribute in that way if and when you have the time!

Smile,
Juliette

/cc @defries

Contributor

jrfnl commented Mar 5, 2014

@deinezauber Hi Nicola, if you go to http://translate.yoast.com/projects/wordpress-seo, you can register and contribute to the translation. The current Italian translation is incomplete and some of the completed phrase might need checking, so it'd be great if you contribute in that way if and when you have the time!

Smile,
Juliette

/cc @defries

barrykooij added a commit that referenced this issue Mar 16, 2014

This should fix the keyword density problem in the Page Analysis.
Related #707
Related #720
Related #349
Related #729
Related #703
Related #264
Related #756
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment