-
-
Notifications
You must be signed in to change notification settings - Fork 334
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inclusion of the TopicRank Keyphrase Extraction algorithm #208
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
But not on the chunks themselves! I'm not sure what behaviour is preferred here
…ce tests from another
…ature/topicrank
Friendly ping on this! @ceteri |
ceteri
approved these changes
Mar 6, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Outstanding! Many thanks @tomaarsen
We'll need to add more about TopicRank
in the docs plus a notebook example.
The improvements on testing look great!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello!
As promised: a beefier PR.
Pull request overview
TopicRank
andTopicRankFactory
class.Details on TopicRank
TopicRank was mentioned in the useful discussion in #174 by @BrandonKMLee and @louisguitton, and was originally introduced in a paper by (Bougouin et al., 2013). It focuses on keyphrase extraction, and relies on TextRank as a part of the overall algorithm. The paper is quite interesting, so I would recommend looking at it, but I'll quickly summarize the algorithm that is presented (copied from my comment in the code):
After this stage, we have preprocessed text.
After this stage, we have a list of keyphrases that may be topics.
linking using simple set-based overlap of lemmas. Similarity is achieved at > 25%
overlap. Note: PyTextRank deviates from the original algorithm here, which uses
stems rather than lemmas.
After this stage, we have a list of topics.
(i.e. clusters derived in the last step), where edge weights are higher between
topics that appear closer together within the document.
After this stage, we have a ranked list of topics.
represent that topic.
After this stage, we have a ranked list of topics, with a keyphrase to represent
the topic.
Sample usage & output
The
TopicRank
algorithm implemented in this PR can be used in essentially the same way as TextRank:This outputs:
To compare, this is the output of the original https://github.com/adrien-bougouin/KeyBench/ repository is as follows, when given the same input:
This output is also given by
pke
, by the same author:Full transparancy: As you can see, there are some small differences between this TopicRank implementation and the originals. E.g.
numerical solution methods
ranks lower, andexchanger compression
ranks higher. I can try to invest a bit more time to try and debug this, but I suspect that this is caused by a differint way to find potential candidates and differing TextRank parameters.Lastly, for context, replacing
nlp.add_pipe("topicrank")
withnlp.add_pipe("textrank")
gives the following results:Implementation details
New default config
pytextrank/__init__.py
has been given a new configuration dict:_TOPIC_DEFAULT_CONFIG
. This configuration dict is simply an extension of the existing_DEFAULT_CONFIG
, with thethreshold
andmethod
attributes added. In this file, a component factory for theTopicRank
class is added.New classes
Both a
TopicRankFactory
andTopicRank
class were implemented, following the design structure of the previously created classes and factories, with the addition of the aforementionedthreshold
andmethod
attributes.New attributes
The
threshold
attribute is a float between 0 and 1, which is used in the TopicRank candidate clustering algorithm to specify the minimum similarity between two keyphrases, for them to be eligible for clustering. The original paper uses a value of 0.25 (i.e. 25% similarity of overlap of lemmas), and so this PR also uses 0.25 as a default.The
method
attribute is the string name of a clustering method in candidate clustering. This PR relies onscipy.cluster.hierarchy.linkage
for the hierarchical agglomerative clustering of candidates, and allows users of PTR to specify which clustering method to use. Again, we use the clustering method from the paper as default: "average".New methods
The
TopicRank
class is a subclass ofTextRank
, (although it may cause some confusion, as theTopicRank
implementation also uses thelemma_graph
object fromTextRank
, but TopicRank uses topic graphs instead). The implementation relies on the overriding of some key methods and properties:node_list
: Now returns a cached (!) list of clustered keyphrase candidates, instead of lemmas.edge_list
: Now returns a list representing edges for a complete, weighted graph between the clustered keyphrase candidates.calc_textrank
: Quite similar to the original, but skips out on some of the post-processing.reset
: Also remove thenode_list
caching.And two new methods:
_get_candidates
and_cluster
, which perform steps 2 and 3 of the TopicRank algorithm, respectively (Candidate extraction & Candidate clustering). Note that_get_candidates
is almost justself.doc.noun_chunks
, but it strips undesired (i.e.not self._keep_token(token)
) tokens from the candidate keyphrase.Note that this PR relies on
scipy
andnetworkx
, both of which were already dependencies, so this doesn't introduce any additional ones.Tests
The tests are based on
test_base.py
, but adapted for TopicRank instead. However, some tests for differentthreshold
andmethod
values are still missing.Other changes
conftest.py
changesEach of the fixtures are now limited to
module
, as thenlp.add_pipe
from certain tests was preserved between all tests, which led to undesired behaviour (i.e. multiple instances of PTR in the pipeline)CI
The continuous integration was updated in this PR. I understand that this is kind of separate to the PR itself, but I felt that the additional tests help give confidence in the correctness. In short, the CI now first calls
pre-commit
, and only if that passes, it will runpytest
for Python 3.7, 3.8, 3.9 and 3.10.Quirks
There are some implementation details that might require some discussion, and could have different implementations depending on preferences. I'll mention them here, and I'd like to have some discussion about them.
scrubber
attribute is used once, right here: https://github.com/tomaarsen/pytextrank/blob/df5833e463160c740f8d4a920129070e26756950/pytextrank/topicrank.py#L311-L331The scrubber is applied only on the Phrase
text
, i.e. the keyphrase that "represents" the phrase as a whole. An alternative is to apply the scrubber on all keyphrase candidates._get_candidates
,self._keep_token
is only called on the first tokens of a noun chunk, potentially removing those tokens from the noun chunk. If there is a token that we wish to remove at the middle or end of a noun chunk, then it won't be removed. In practice this won't happen often, as it's mostly meant to remove "The" from "The system" etc., but this functionality might be preferred? The current implementation is simpler however, as it doesn't require splitting noun chunks into multiple keyphrase candidates if there is a undesirable token in the middle. There is also an argument to be made that we usually want to remove certain tokens, but perhaps not if they occur in the middle of an interesting noun chunk. (i.e. that would be the current implementation)TopicRank
extendsTextRank
, but there are some definite differences between them, the inheritedtoken_lookback
attribute ofTopicRank
is unused. Perhaps a warning if a non-default value is used here would be proper? Alternatively, we could remove this attribute from theTopicRank
implementation somehow.What now?
There are still some TODO's, here are a few:
threshold
andmethod
.topicrank.py
, there is still a TODO. On e.g.biasedrank.py
, this place contains a link to https://derwen.ai/docs/ptr/biblio/#kazemi-etal-2020-biased, with the citation for that work. I feel it would be proper to add a citation in the documentation, with a link from the code for TopicRank too. However, I haven't done this as of now.TopicRank
in PTR, although I don't this has the highest priority. After all, to the end user it'll work almost the same as any of the other ranking components regardless.I'd love to hear your thoughts, and if you have any questions or comments, let me know (either here or via Slack). I also have a question for you: is there a specific formatter that you use, that I should use for this work?