Add more options to language selection #18538
Comments
|
We support all ISO-639-1 languages, plus a few ISO-639-3 languages added on demand. Bear in mind that the fediverse does not have enough people for granular language subdivisions to make sense, since it would cut your audience to miniscule proportions. If there is an active community of a specific language that is not represented in the list, then I will add it. |
|
Thanks for the reply :-)
By ‘granular language subdivisions’ you mean things like subdividing similar and mutually intelligible dialects? That is truly counter-productive. What I was talking about is adding distinct languages which are missing. For example, according to this, Nigerian Pidgin has 120 million speakers but no ISO-639-1 code; so do Cantonese (over 80 million speakers), Bhojpuri (52.5 million speakers) and other languages. Limiting one’s feed to a set of languages is one use of language tagging. Others include proper settings for screen readers, searching for toots in a certain languages (for example, in order to find other speakers of said language), language-dependent typographical settings, machine translation, etc. These depend on having the correct language tagged. |
I'm just dropping by here to remind everyone that there is an ISO-639-3 code for that:
This also has an ISO-639-3 code: |
|
Probably related to this: #8933 It would be great if there's a distinction between traditional (zh-hant) and simplified (zh-hans) characters. |
|
A related toot: https://skeletons.gay/notice/AJs5T1hEp1Pr6oWlbk |
Isn’t that on a different level, namely that of writing systems, as opposed to languages? |
|
idk, there're different ways to classify them. Personally I just want to be able to filter my timeline, so that it can distinguish between the two. In ubuntu, I think they have Chinese (Traditional), Chinese (Simplified), Chinese (Hongkong). In msft dotnet they have:
|
|
Some languages have their local varieties (for example: en-US and en-UK), and some languages using double or multiple writing systems (eg. Mongolian, Hokkien/Taiwanese/Min Nan, Mandarin, Aramaic languages). |
|
Recently, I thought it may be help to let the admin to add customized language code or delete it manually. for example: For a Taiwan Instance: The filter logic can be: |
|
It would be great to have separated options at least for Cultures for these users is really different from |
|
We had been using automated language detection using cld3 on posts for many years, which was often wildly inaccurate (the main reason it has now been removed), but even in the best case, you could not expect it to distinguish correctly between regional variants of languages ( Even though that was the main reason for the design, I think it still makes sense. I perfectly understand that not every variant of a language is mutually intelligible, but I maintain that:
That is to say, I think that sticking to |
|
CLDR is a more sensible source for locales to support. https://cldr.unicode.org/ At a minimum you need to have a name for the language code, if you want to offer it in an interface. And you need to know at least some basics like script and directionality if you need to use the language code to define your HTML output. |
No, according to your opinion, I still don't understand the reason to limit user from having a choice to do what they want to do. And there's already a language filter in preference which can perfectly avoid the problem you are concerning, what on earth are you talking about?
Following your logic, I strongly recommend that Mastodon should also remove Ukrainian from languages list and merge them into Russian because there are more filtered results for Ukrainian & Russian speakers. I even wrote an user script to batch adding ~600 filters (because Mastodon's filter still doesn't support RegExp) to filter out annoying zh_CN messages. Looks so delicate, doesn't it?
|
If users filter something, to the point that nothing gets shown on their timeline, then that's the filter working as intended. It's their choice, that's why the users use filter in the first place. Maybe it's not a good thing for mastodon.social, that are trying to attract as many new users as it can. I understand that new users will tend to stay if the timeline has many contents and there're a lot of engagements. But not all instances are trying to become mastodon.social. |
|
Il 08/09/22 00:25, onmyouji ha scritto:
If users filter something, to the point that nothing gets shown on their timeline, then that's the filter working as intended.
Small note on this: there's no clear indication of how many posts are
filtered out by the language filtering preferences. So it cannot be
assumed that the user knows why they're seeing the amount of posts
they're seeing.
|
Sorry, can you explain more about that statement? Let's say a user join an instance, they think the local timeline is too crowded with posts from different languages that they don't want to see. They go to their language filter settings, and tick Spanish and Korean. When they go back to the local timeline, it shows a lot fewer posts (or maybe nothing). So you're telling me, the user won't be able to know why the instance they joined suddenly becomes a dead town, after they specifically choose to filter the language? I mean, that's what the filter is for right? To cut the amount of content the users see, to choose specifically what content (language) should appear in the timeline. |
|
Il 09/09/22 11:52, onmyouji ha scritto:
So you're telling me, the user won't be able to know why the instance they joined suddenly becomes a dead town, after they specifically choose to filter the language?
Your scenario requires that the user compares the same timeline, with
and without the setting, in short order after changing the settings.
When time passes, if a different setting becomes more suitable for the
user, the user would need to go through several steps:
1) realise that something is missing from the timeline (this might
happen because other users are referencing something invisible; perhaps
a trending hashtag which returns nothing);
2) remember that the language filter exists;
3) remember that they set the language filter;
4) guess that they might have set the filter to languages likely to not
cover the invisible content;
5) find the language filter in the settings [I often forget where it is];
6) change the setting and go back to the search previously performed in
time to see that the results are different;
7) tweak the settings until satisfaction.
I have a few languages set in my filter and I wouldn't remember which
languages they are. Moreover, I have no idea what's the language in
which the toots I see are classified under, let alone the toots I don't
see. Moreover, the language can change without the users even knowing:
for instance now the default is the interface language, and I have no
idea what interface language other users are using. If a user suddenly
switches to posting to some other language I had not thought, they'll
just disappear from my timeline and I'll never remember to check their
content with different settings.
|
|
I think it's very easy and important to add the special ISO 631-3 language codes:
As for how it interacts with the filter language option, they would all be a separate subsetting so these codes will be enabled by default even for users that already have a filter language setting. This sounds like a good first issue. Can I try to implement it with some hope of it being merged? |
|
As for some of the sublanguage issues brought by @Gargron @poga @Yoxem @onmyouji and others, I think that the ideal way forward is to treat languages as "hierarchical" things" with tri-state checkboxes for all non-leaf settings. For Portuguese (which I do speak) it would be like: Note: this list came from the member states/regions of the CPLP - Comunidade de Países de Língua Portuguesa (Lusophone Commonwealth) and is roughly in order of speakers. Note: I only included the ISO codes here to make myself clear but I don't think that they should be exposed to the end user, at least by default So a setting for Chinese wouldn't be a simple checkbox but a tree of (msotly tri-state) checkboxes like: Note: I don't speak any Chinese language(s) so I'm surely exaggerating and undergerating with some of these options. To avoid making the UI too intimidating, one option would be to let the user manually add countries (with a dropdown selector) or codes ot their filter languages setting. Something like: |
Don't bother, just save your time. If you've been following mastodon since the beginning, you'll know that it won't get merged. Everyone and every major organization on the planet (e.g. Linux, Steam, Twitter, Microsoft etc) knows to make a distinction between Traditional and Simplified Chinese, it's just common sense. But the fact that in year 2022, users still have to beg him for it should explain to you how reasonable he is. |
Yeah, cannot agree with you anymore. I've felt so ridiculous and tired on such topic (as a Taiwanese, we are forced to face similar situations everyday), as if I'm just keeping explaining something to some Pro-China who always give runaround and just mean to force Taiwanese to use their language. |
|
As a Hongkonger, I agree with @kuanyui too. |
|
As a multilingual Taiwanese, I want to avoid seeing any toots from China. I would need to set up this long filter to get rid of those characters (and being carefully not to block Japanese kanji accidentally). I read toots in English, Japanese, Traditional Chinese (TW/HK). Please do not force users from different countries using the same language, as @dollars0427 said.
|
|
BTW, for the non-Chinese users, I can tell you, even Just like both Italian and English use Latin alphabets but they cannot understand each other. |
|
It seems no sign of an official solution to this topic in seeable future, so provide the User Script I had written before, which can batch install lots of filters, as a workaround for those who feel tired to this world. https://gist.github.com/kuanyui/b8afc959cb4c17d17b45c6ae0669a2ec |
|
After read all the comments, I found that the language options actually serve two distant functions that (some) users want.
Many of those debates around
They can read them, but they want use language option to avoid those "Pro-China" ppl. Another example:
Again, they can understand those toots, but want to use language as a proxy to filtered out toots from China. In fact, Mandarin Chinese written in simplified/traditional script ARE inter-intelligible (with context and a little guessing if they never see the other variant). The need of distinguishing between them is likely fall into category 2. However, the category 1 problem indeed exists around |
|
@sorz, correct. Regardless either 1 or 2, it should be understandable (and should be respected) that one would like to avoid contents they do not enjoy when browsing their timelines on Mastodon. This is undoubtedly the point of having filters around. A language always carries culture differences and implicit references beyond itself, the (true) meanings of a particular language represent the culture and values of a particular social group. Forcing different social groups to be recognized under the same |
|
"Chinese" is a language family, on the same level of granularity as "Romance" or "Germanic" or "Slavic". I would like to be able to select Hokkien (ISO 639-3: nan). This is regardless of geographical region or writing system. Some people write romanised Hokkien (the most common systems being POJ and Tailo), and some people write Chinese characters (several systems exist, usually with traditional characters). For more information, see: https://en.wikipedia.org/wiki/Written_Hokkien Insisting on "zh" for Hokkien and other non-Mandarin Chinese languages is extremely offputting, and perpetuates the marginalisation of these languages.
This does not seem like a welcoming attitude towards minority languages. But taking this at face value, I would say that the most vibrant online Hokkien community is probably the Taiwanese Hokkien community (where the language is often referred to as simply "Taiwanese"). For some examples of Hokkien toots, see: https://g0v.social/tags/Taigi -- I hope this is evidence enough that this language should be supported. |
|
It seems that very small communities can indeed get their languages included in the dropdown list: |
|
Yes, that seems really strange. I made the leap over from Twitter today and when I migrated my English/Cantonese account (I have several which I separate by languages) to my dismay I saw that the only way I can tag my Cantonese posts is either to mis-label them English or mis-label them 中文, never mind not being able to filter out Mainland stuff in 中文 I have no interest in. It's not a zero sum game, I think it's great people can choose Navajo, Inuktitut and tokipona (had to Google that one) but it seems like a list that is grossly skewed (though I couldn't say what to, perhaps languages the original devs were interested in?) I like the idea of hierarchical nesting, it makes sense as there are a lot of languages. Surely Mastodon can't possible be looking to be more linguistically restrictive than Twitter? |
@Gargron The new ISO 639 is just out with substantial enlargement, so I can back up my opinion below with the official evidence. While the full text is proprietary, please allow me to cite its section 6.2.1:
which essentially means that they publicly admit grouping up several practically unintelligible "dialects" into an identical ISO 639 code (the situation has always existed but now explicitly ratified). From the backstage perspective, this assumes the existence of other methods to specify subdivisions, such as IETF language tag or upcoming ISO 21636 framework, so in such cases we will need the help of combined language tags.
In my observation, this problem affects more the users of some of the "big languages", including Chinese in this thread, or Arabic, where they nominally believe they all still speak "Latin" but actually not. |
|
With regard to zxx, while an image might not have a language, its alternative text would. I know not all images have alternative text, but they are supposed to have it. Similarly, emojis tend to have English Unicode names (which I suppose might have translations, but I don't know that). At work, my colleague who was responsible for setting standards for Chinese translation told me that while people who normally use Simplified Chinese (zh-HANS) can usually also read Traditional Chinese (zh-HANT), that the reverse is not true. So we always use zh-HANT. |
|
@ChasBelov no, this is not true. Many Simplified Chinese users don't read Traditional Chinese, they may not be familiar with the characters, phrases, etc. Your colleagues may be able to recognize both, I believe this is not common among Chinese language family users. Again, please try to be respectful of the culture/political differences to all social groups. We are all different. :) |
|
@ChasBelov It depends (mostly on their educational background). A more accurate metaphor about the Simplified/Traditional Chinese situation is that: Trad. characters looks to Simp. users 𝔅𝔩𝔞𝔠𝔨𝔩𝔢𝔱𝔱𝔢𝔯, while Simp. characters to Trad. users something like Unifon. So it is probably easier for Simp. user to recognize Trad. text (as the more they learn Classical Chinese) than the other way around, but for most ordinary people, they are just guesswork either way. |
|
Recently updated my Simplified Chinese filter, feel free to apply to your timeline 😃 This should cover a good percentage of Simplified Chinese toots, without filtering out Japanese 漢字. |
Pitch
Currently the new language selection dropdown tool (introduced in #18420) supports a limited number of languages. I suggest:
¹ How does language tagging works across the Fediverse? Is there any standard for language codes? This should be taken into consideration.
Motivation
Mainly two groups of users will benefit from the suggestion:
The text was updated successfully, but these errors were encountered: