Skip to content
This repository has been archived by the owner on Oct 4, 2022. It is now read-only.

Introduce stemming for Internal linking suggestions and Insights #186

Closed
wants to merge 154 commits into from

Conversation

nataliashitova
Copy link
Contributor

@nataliashitova nataliashitova commented Apr 2, 2019

Summary

Package(s) involved: yoastseo

  • Should the change be included in the package changelog?
    • No
    • Yes
  • Should the change be included in one or more plugin changelogs?
    • No
    • Free
    • Premium
    • Other (please specify)
  • Package changelog item (if applicable):
    • Uses feature-flag functionality to determine if the old or the new analysis should be applied.
    • In the new analysis, changes the way prominent words are extracted for the Internal linking suggestions and Insights features. No multiple-word combinations are considered anymore, but single words only. For languages with morphological support (English and German) different morphological forms of words (e.g., link, linking, links, linked) are recognized and collapsed together, which improves the performance of the said features.
    • In the new analysis, changes the Insights functionality in the following way: no words that occurred less than 5 times in the text of the post are displayed. For languages with morphological support (for now, English and German) different word forms of the same word are collapsed based on their stem/base form (e.g., the stem of words link, linking, links, and linked is link). If the stem coincides with an actual word used in the text, it is displayed to the user in the list of Insights. If the stem does not coincide with any words used in the text, the first word form of the stem that occurred in the text will be used for display.
    • In the new analysis, changes the Internal linking suggestions functionality in the following way: not only the text of the post but also different meta data fields are used for analysis, i.e., title, keyphrase, synonyms, meta description, subheadings (H2 and H3 levels). One occurrence of a word in the said meta data fields is deemed more important than one occurrence of this word inside the text. Stems are further saved in the database and used for matching of prominent words between posts and the internal linking functionality per se.
  • Plugin changelog item (if applicable): Introduces stemming for the calculation of the prominent words, used in Internal linking suggestions and in Insights.

Package(s) involved: components

  • Should the change be included in the package changelog?
    • No
    • Yes
  • Should the change be included in one or more plugin changelogs?
    • No
    • Free
    • Premium
    • Other (please specify)
  • Package changelog item (if applicable):
    • Uses feature-flag functionality to determine whether a single word or a multi-word combination shall be returned in the KeywordSuggestions component
  • Plugin changelog item (if applicable):

Test instructions

This PR can be tested by following these steps:

Impact check

  • This PR affects the following parts of the plugin, which may require extra testing:
    • Internal linking suggestions
    • Insights

UI changes

  • This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Quality assurance

  • I have tested this code to the best of my abilities
  • I have added unittests to verify the code works as intended

Fixes #79

nataliashitova and others added 30 commits February 11, 2019 16:11
After a small discussion with the team, we decided to disable this rule.
As this is our development tool and we do not want to put to much time into optimizing the coding standards within.
…rnal-linking

Add paper attributes to internal linking research
nataliashitova and others added 16 commits May 15, 2019 10:06
…o that they don't import the morphology file directly, but do so via getMorphologyData
…282)

Use getMorphologyData in specs and add -v2 to English morphologyData
 - @yoast/example-component-app@1.0.20
 - @yoast/content-analysis-app@0.1.19
 - @yoast/algolia-search-box@1.3.0-rc.0
 - @yoast/analysis-report@0.4.0-rc.0
 - @yoast/components@0.4.0-rc.0
 - @yoast/configuration-wizard@1.3.0-rc.0
 - eslint-config-yoast@5.0.15
 - @yoast/feature-flag@0.3.0-rc.0
 - @yoast/helpers@0.4.0-rc.0
 - @yoast/search-metadata-previews@1.3.0-rc.0
 - @yoast/social-metadata-previews@0.4.0-rc.0
 - @yoast/style-guide@0.4.0-rc.0
 - yoast-components@4.27.0-rc.0
 - yoast-social-previews@1.4.9-rc.0
 - yoastseo@1.54.0-rc.0
…t into fix-content-analysis-app-morphology-data-imports
…thub.com:Yoast/javascript into fix-content-analysis-app-morphology-data-imports
…y-data-imports

Fix content analysis app morphology data imports
@manuelaugustin manuelaugustin added this to the 11.4/multiple milestone May 28, 2019
@manuelaugustin
Copy link
Contributor

Closing this PR as we had already created a release branch based on the feature branch.

@manuelaugustin manuelaugustin deleted the feature/internal-linking branch June 19, 2019 09:07
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New internal linking mechanism should use stems
5 participants