Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new filter is not compatitle with languages that normally do not have space between words #19080

Open
ooozerooo opened this issue Aug 29, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@ooozerooo
Copy link

ooozerooo commented Aug 29, 2022

Steps to reproduce the problem

  1. add a Japanese/Chinese phrase(more than one character) as a keyword into the filter and tick the 'whole word' box
  2. post a new toot contains the phrase

Expected behaviour

the post will be filtered (what the old filter version does)

Actual behaviour

the post will not be filtered and shows normally on timeline

Specifications

It seems like the new filter only recognizes characters between space as 'word', however, this is not true with languages like Japanese, Chinese and many else. Basically the new filter design will cause keywords that involve languages which normally do not use space between words to have no effect at all.

For instance, if I don't want to see the word 'Beijing(北京)' on my timeline, add '北京 ' to the filter does nothing. Because Chinese and Japanese users wouldn't say 'I went to Beijing' as ‘我去了(space)北京' or '北京(space)に行った`, the filter is not doing any help.

@ooozerooo ooozerooo added the bug Something isn't working label Aug 29, 2022
@ooozerooo
Copy link
Author

I have double checked and it seems like unticking the 'whole word' box can solve this, but it's still different from the old version. In the old version if you tick 'whole word' for filtering languages like Japanese and Chinese that normally do not involve the use of space the filter still works. Therefore migrating from the old to new version will cause some filters to become ineffective, which may lead to unprepared people being exposed to unwanted content, and is better solved before official release.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant