Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solving the issue "Template matching regex features only matches prefix" #170

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

Sahethi
Copy link

@Sahethi Sahethi commented Jan 21, 2022

Heyy, I tried to solve this issue referring https://phabricator.wikimedia.org/T250522

I initially did want to pass a param to the template_names_matching() method but I believe it would have demanded changes in the definition in revscoring and passing a delimiter would still require the same amount of work, hence I individually added $ to every regex which called template_names_matching().

@halfak
Copy link
Member

halfak commented Jan 21, 2022

Can you add some tests to demonstrate the problem this solves?

@Sahethi
Copy link
Author

Sahethi commented Jan 21, 2022

@halfak Okay I'll try doing that

CN_TEMPLATES = [
r"Citation[_ ]needed",
r"Cn",
r"Fact"
]
cn_templates = wikitext.revision.template_names_matching(
"|".join(CN_TEMPLATES), name="enwiki.revision.cn_templates")
"$|".join(CN_TEMPLATES)+"$", name="enwiki.revision.cn_templates")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered a regex in the format (?:foo|bar|baz)$, with a single $ after the parenthesis?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay I'll try to check that

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey, are you indicating a case scenario where we would have a regex as
Citation[_ ]needed$|Cn$|Fact$|(?:foo|bar|baz)$$ causing a double $

I've tired to check if it works as below
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, not really. I was thinking in something like this:
(?:Citation[_ ]needed|Cn|Fact)$
See https://regex101.com/r/GinCwl/1.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, not really. I was thinking in something like this: (?:Citation[_ ]needed|Cn|Fact)$ See https://regex101.com/r/GinCwl/1.

Oh that does seem much better, thank you.
I'll do that and run some tests

@Sahethi
Copy link
Author

Sahethi commented Feb 11, 2022

Can you add some tests to demonstrate the problem this solves?

Hey sorry about getting back late. But I have been trying to match a regex pattern with the template and for some reason even for the pattern which doesn't match, the test doesn't seem to fail.

I have changed the code in articlequality/feature_lists/ptwiki.py to the following (only cn_templates is modified)
image

and the test on cn_templates still doesn't fail (test_cn_templates from articlequality/tests/feature_lists/test_ptwiki.py)
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants