Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch and show pronouns for chatters. #5442

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

DeltaTimo
Copy link

Common pronoun providers like alejo.io and pronoundb.org have respective browser extensions for twitch, but so far Chatterino does not support these providers.

I have now implemented a provider that fetches pronouns from APIs.
image

Pronoun providers can be configured in code, where, if multiple are configured the latter ones serve as a fallback for the former ones.

Right now, pronoundb.org and pronouns.alejo.io are implemented.

Pronouns are fetched when a message from a new user arrives and is cached application wide per session (as opposed to cache per tab). Pronouns are not added to the first message from a user. In the future, pronouns could be added to previous messages too.

For future-proofing, it is possible to fetch pronouns for multiple users at once, but right now, pronouns are fetched per-message if required.

@pajlada pajlada self-assigned this Jun 6, 2024
@pajlada
Copy link
Member

pajlada commented Jun 6, 2024

Some comments before I make a proper review:
Have you reached out to the API owners regarding adding this feature?
Do those API owners have some sort of privacy policy regarding data they store, and for the requests we make from Chatterino?
I'm not super fond of a request being spun off as soon as a message comes in, would it be super bad if it instead runs on a timer & bulk fetches data every ~30 seconds or so?

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 56. Check the log or trigger a new build to see more.


namespace chatterino {

class IPronounsApi {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: destructor of 'IPronounsApi' is public and non-virtual [cppcoreguidelines-virtual-class-destructor]

class IPronounsApi {
      ^
Additional context

src/providers/pronouns/IPronounsApi.hpp:12: make it public and virtual

class IPronounsApi {
      ^

using RequestT = PronounsApiRequest<ResultT>;

// Fetch pronouns from the API.
virtual void fetch(std::vector<PronounUser> users, RequestT::CallbackT &&onDone) {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: rvalue reference parameter 'onDone' is never moved from inside the function body [cppcoreguidelines-rvalue-reference-param-not-moved]

  virtual void fetch(std::vector<PronounUser> users, RequestT::CallbackT &&onDone) {};
                                                                           ^

using RequestT = PronounsApiRequest<ResultT>;

// Fetch pronouns from the API.
virtual void fetch(std::vector<PronounUser> users, RequestT::CallbackT &&onDone) {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the parameter 'users' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param]

  virtual void fetch(std::vector<PronounUser> users, RequestT::CallbackT &&onDone) {};
                                              ^


namespace chatterino {

PronounUser::PronounUser(): id{}, username{} {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for member 'id' is redundant [readability-redundant-member-init]

Suggested change
PronounUser::PronounUser(): id{}, username{} {}
PronounUser::PronounUser(): , username{} {}


namespace chatterino {

PronounUser::PronounUser(): id{}, username{} {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for member 'username' is redundant [readability-redundant-member-init]

Suggested change
PronounUser::PronounUser(): id{}, username{} {}
PronounUser::PronounUser(): id{}, {}

src/providers/pronouns/UserPronouns.hpp Show resolved Hide resolved
{"ziehir", "zie/hir"},
};

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonArray array) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'parse' can be made static [readability-convert-member-functions-to-static]

Suggested change
/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonArray array) {
/*static*/ static UserPronouns PronounsAlejoApi::parse(QJsonArray array) {

{"ziehir", "zie/hir"},
};

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonArray array) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: out-of-line definition of 'parse' does not match any declaration in 'chatterino::PronounsAlejoApi' [clang-diagnostic-error]

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonArray array) {
                                          ^

{"ziehir", "zie/hir"},
};

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonArray array) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the parameter 'array' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param]

Suggested change
/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonArray array) {
/*static*/ UserPronouns PronounsAlejoApi::parse(const QJsonArray& array) {

return {};
}

if (array.size() == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty]

Suggested change
if (array.size() == 0) {
if (array.empty()) {
Additional context

.qtinstall/Qt/6.6.2/gcc_64/include/QtCore/qjsonarray.h:209: method 'QJsonArray'::empty() defined here

    inline bool empty() const { return isEmpty(); }
                ^

@DeltaTimo
Copy link
Author

Some comments before I make a proper review: Have you reached out to the API owners regarding adding this feature? Do those API owners have some sort of privacy policy regarding data they store, and for the requests we make from Chatterino? I'm not super fond of a request being spun off as soon as a message comes in, would it be super bad if it instead runs on a timer & bulk fetches data every ~30 seconds or so?

I think it should be easily possible to fetch on a timer, although I'm not familiar enough with the code base to know how or where timers are usually registered and run. Could you give me a pointer to where timers are usually placed? There was a "recent chatters" variable somewhere iirc, we could use this to occasionally on timer fetch pronouns for those.

Alejo.io unfortunately, to my understanding, has no documentation for the API at all, PronounDB has a documentation at https://pronoundb.org/wiki/api-docs and mentions setting a User-Agent (which should be set, as NetworkRequest is used). It does also provide a contact email, I'm probably going to ask there on what's the preferred way.

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 56. Check the log or trigger a new build to see more.

I just realized I read and then completely skipped the entire clang-tidy part in the contribution guide to "do it later"... Whoops, I shall run it again locally and make the changes.

It also seems some things are broken on windows build (I think it's some std::size_t thing). I'll see if I can fix that, although I don't have Windows on any machine here.

@DeltaTimo DeltaTimo changed the title feat: Fetch and show pronouns for chatters. Draft: feat: Fetch and show pronouns for chatters. Jun 6, 2024
@DeltaTimo DeltaTimo marked this pull request as draft June 6, 2024 17:11
@DeltaTimo DeltaTimo changed the title Draft: feat: Fetch and show pronouns for chatters. feat: Fetch and show pronouns for chatters. Jun 6, 2024
@pajlada
Copy link
Member

pajlada commented Jun 6, 2024

Any sort of nitpicks or windows/macos/linux-specific things you can't fix can be easily fixed by any of our other contributors as part of the review process - we don't 100% listen to clang-tidy, so you can make a judgement call whether you care to listen to it

For the timer thing, I can bring up some example when I do my review of this PR in the weekend.
Generally, my opinion of the feature is that it should be opt-in as a user of Chatterino to use the API, and generally we shouldn't modify old messages unless very necessary.

@DeltaTimo
Copy link
Author

Any sort of nitpicks or windows/macos/linux-specific things you can't fix can be easily fixed by any of our other contributors as part of the review process - we don't 100% listen to clang-tidy, so you can make a judgement call whether you care to listen to it

Alright that sounds good! I'll go through the code again later or tomorrow.

For the timer thing, I can bring up some example when I do my review of this PR in the weekend.

Sounds good! When will you be reviewing? I might force-push new changes to my branch, but if needed I can just create a new branch and work on that.

Generally, my opinion of the feature is that it should be opt-in as a user of Chatterino to use the API, and generally we shouldn't modify old messages unless very necessary.

I've already created a checkbox in the settings menu to opt-out, I'll default it to disable then, which should also disable fetching.

@pajlada
Copy link
Member

pajlada commented Jun 6, 2024

I should be able to get a full review in on Saturday on Sunday on my stream, can delay to Sunday (or to whenever) if you want more time before I take a look

We Squash & Merge here, so you don't have to keep commits clean, I just prefer no force pushes after I've started reviewing

@Nerixyz
Copy link
Contributor

Nerixyz commented Jun 6, 2024

To fix compilation on Windows, you need to add an l, because long and unsigned long are four bytes (reference):

diff --git a/src/providers/pronouns/pronoundb/PronounsPronounDbApi.cpp b/src/providers/pronouns/pronoundb/PronounsPronounDbApi.cpp
index ca39f70f..728aed67 100644
--- a/src/providers/pronouns/pronoundb/PronounsPronounDbApi.cpp
+++ b/src/providers/pronouns/pronoundb/PronounsPronounDbApi.cpp
@@ -111,7 +111,7 @@ void PronounsPronounDbApi::fetch(std::vector<PronounUser> allUsers, IPronounsApi
   auto request = std::make_shared<IPronounsApi::RequestT>(allUsers.size(), std::move(onDone));
 
   for (std::size_t i {0}; i < allUsers.size(); i += 50) {
-    auto end = static_cast<std::size_t>(std::min(allUsers.size() - i, 50ul));
+    auto end = std::min(allUsers.size() - i, 50ull);
     auto users = std::vector<PronounUser>(allUsers.begin() + i, allUsers.begin() + i + end);
 
     NetworkRequest(PronounsPronounDbApi::makeRequest(users))

Just some general comments:

  • Please use clang-format to format your code (tip: turn on format-on-save in your editor).
  • The posted clang-tidy suggestions are valid (maybe the enum size one is debatable). You can use clang-tidy through clangd in your editor.
  • For documentation comments, use triple slash comments (/// - technically you could also use /** */, but it wastes two lines per comment).
  • Use QString over std::string, because we use it everywhere, and it's reference counted (but it doesn't have SSO sadly)
  • Although we mostly use one top-level namespace, I feel like a separate namespace for pronouns could be nice (so something like chatterino::PronounsPronounDbApi could become chatterino::pronouns::DbApi)
  • Currently, Chatterino makes one request to the DB per user, which could spam the provider quite a bit in fast chats. Maybe we could batch some requests (maybe at most one request per two seconds?).

@pajlada
Copy link
Member

pajlada commented Jun 6, 2024

For documentation comments, use triple slash comments (/// - technically you could also use /** */, but it wastes two lines per comment).

Although we mostly use one top-level namespace, I feel like a separate namespace for pronouns could be nice (so something like chatterino::PronounsPronounDbApi could become chatterino::pronouns::DbApi)

You can (and probably should) leave these changes until the implementation has been reviewed

Currently, Chatterino makes one request to the DB per user, which could spam the provider quite a bit in fast chats. Maybe we could batch some requests (maybe at most one request per two seconds?).

this is the same as i mentioned above

@DeltaTimo DeltaTimo force-pushed the mr/chatterino2/add-pronoun-provider branch 2 times, most recently from 488b6a7 to d72b09c Compare June 6, 2024 21:16
@DeltaTimo
Copy link
Author

I just noticed both of you had commented since I last checked (and before I pushed)!

@Nerixyz will do!

The latest version now queues users for fetching when messages arrive and are regularly dequeued and fetched in a batch. I don't think the alejo.io API supports batch requests and the (open-source) browser extension fetches per message too, but I still added a limit to avoid completely flooding it.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 55. Check the log or trigger a new build to see more.


namespace chatterino {

class IPronounsApi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: destructor of 'IPronounsApi' is public and non-virtual [cppcoreguidelines-virtual-class-destructor]

class IPronounsApi
      ^
Additional context

src/providers/pronouns/IPronounsApi.hpp:13: make it public and virtual

class IPronounsApi
      ^

using RequestT = PronounsApiRequest<ResultT>;

// Fetch pronouns from the API.
virtual void fetch(std::vector<PronounUser> users,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the parameter 'users' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param]

    virtual void fetch(std::vector<PronounUser> users,
                                                ^


// Fetch pronouns from the API.
virtual void fetch(std::vector<PronounUser> users,
RequestT::CallbackT &&onDone){};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: rvalue reference parameter 'onDone' is never moved from inside the function body [cppcoreguidelines-rvalue-reference-param-not-moved]

                       RequestT::CallbackT &&onDone){};
                                             ^

namespace chatterino {

PronounUser::PronounUser()
: id{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for member 'id' is redundant [readability-redundant-member-init]

Suggested change
: id{}
:


PronounUser::PronounUser()
: id{}
, username{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for member 'username' is redundant [readability-redundant-member-init]

Suggested change
, username{}
,

}
UserPronouns::UserPronouns(std::string pronouns)
{
if (pronouns.length() == 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the 'empty' method should be used to check for emptiness instead of 'length' [readability-container-size-empty]

Suggested change
if (pronouns.length() == 0)
if (pronouns.empty())
Additional context

/usr/include/c++/13/bits/basic_string.h:1219: method 'basic_string'::empty() defined here

      empty() const _GLIBCXX_NOEXCEPT
      ^

UserPronouns();
UserPronouns(std::string);

inline std::string format() const
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'format' has inline specifier but is implicitly inlined [readability-redundant-inline-specifier]

Suggested change
inline std::string format() const
std::string format() const

src/providers/pronouns/UserPronouns.hpp Show resolved Hide resolved
/*static*/ std::optional<std::unordered_map<std::string, std::string>>
PronounsAlejoApi::pronounsFromId = {};

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonObject object)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'parse' can be made static [readability-convert-member-functions-to-static]

Suggested change
/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonObject object)
/*static*/ static UserPronouns PronounsAlejoApi::parse(QJsonObject object)

/*static*/ std::optional<std::unordered_map<std::string, std::string>>
PronounsAlejoApi::pronounsFromId = {};

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonObject object)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: out-of-line definition of 'parse' does not match any declaration in 'chatterino::PronounsAlejoApi' [clang-diagnostic-error]

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonObject object)
                                          ^

@Felanbird
Copy link
Collaborator

I don't think the alejo.io API supports batch requests and the (open-source) browser extension fetches per message too, but I still added a limit to avoid completely flooding it.

My main issue I outlined in #3346 was that alejo almost certainly can't keep up with the amount of requests we'd send, that 2 different implementations would return a different subset of users with pronouns, and it falls to blaming Chatterino if this feature we add works very sporadically (e.g. user reaction to bttv live emote updates)

pronounsdb does look like it could better handle the brunt of our requests in comparison though

@DeltaTimo
Copy link
Author

I don't think the alejo.io API supports batch requests and the (open-source) browser extension fetches per message too, but I still added a limit to avoid completely flooding it.

My main issue I outlined in #3346 was that alejo almost certainly can't keep up with the amount of requests we'd send, that 2 different implementations would return a different subset of users with pronouns, and it falls to blaming Chatterino if this feature we add works very sporadically (e.g. user reaction to bttv live emote updates)

pronounsdb does look like it could better handle the brunt of our requests in comparison though

PronounDB really does seem to handle it better, but it is unfortunately less widely used at least in the channels I've been lurking in. I'm wondering how alejo.io is working in the first place, as it seems that the browser extension is working in a flooding way too. If this is the intended way, and the way it is already implemented in its own browser extension, I don't see what sets apart Chatterino from just using a browser with the extension enabled (and then flooding the server in very lively channels).

@Felanbird
Copy link
Collaborator

I don't see what sets apart Chatterino from just using a browser with the extension enabled

Browser chat is 1 channel and Chatterino is up to 100

@DeltaTimo
Copy link
Author

I don't see what sets apart Chatterino from just using a browser with the extension enabled

Browser chat is 1 channel and Chatterino is up to 100

True. I suppose only queuing new users on message in the active tab leads to the aforementioned weird sporadic behaviour?

@iProdigy
Copy link
Contributor

iProdigy commented Jun 6, 2024

My recommendation is to only display the pronouns in the usercard - this minimizes http requests and avoids implementation complexity (of batching requests or caching thousands of pronouns)

I understand that the usercard approach reduces the convenience factor for this feature, but the technical challenges (in my opinion) puts us in a situation where we need to apply more pressure on Twitch to natively support pronouns (to display it inline with each chat message). Similar to the no-video and no-audio badges, there should be he,she,they badges natively selectable on Twitch, which would be very easy for us to support in a scalable manner

@Felanbird
Copy link
Collaborator

there should be he,she,they badges natively selectable on Twitch, which would be very easy for us to support in a scalable manner

not even just a badge, pronouns sent in the tags would be fine too (don't care about people who hate tags, they work)

@DeltaTimo DeltaTimo closed this Jun 7, 2024
@DeltaTimo DeltaTimo reopened this Jun 7, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 53. Check the log or trigger a new build to see more.


namespace chatterino {

class IPronounsApi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: destructor of 'IPronounsApi' is public and non-virtual [cppcoreguidelines-virtual-class-destructor]

class IPronounsApi
      ^
Additional context

src/providers/pronouns/IPronounsApi.hpp:13: make it public and virtual

class IPronounsApi
      ^

using RequestT = PronounsApiRequest<ResultT>;

// Fetch pronouns from the API.
virtual void fetch(std::vector<PronounUser> users,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the parameter 'users' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param]

    virtual void fetch(std::vector<PronounUser> users,
                                                ^


// Fetch pronouns from the API.
virtual void fetch(std::vector<PronounUser> users,
RequestT::CallbackT &&onDone){};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: rvalue reference parameter 'onDone' is never moved from inside the function body [cppcoreguidelines-rvalue-reference-param-not-moved]

                       RequestT::CallbackT &&onDone){};
                                             ^

namespace chatterino {

PronounUser::PronounUser()
: id{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for member 'id' is redundant [readability-redundant-member-init]

Suggested change
: id{}
:


PronounUser::PronounUser()
: id{}
, username{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for member 'username' is redundant [readability-redundant-member-init]

Suggested change
, username{}
,

UserPronouns();
UserPronouns(std::string);

inline std::string format() const
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'format' has inline specifier but is implicitly inlined [readability-redundant-inline-specifier]

Suggested change
inline std::string format() const
std::string format() const

src/providers/pronouns/UserPronouns.hpp Show resolved Hide resolved
/*static*/ std::optional<std::unordered_map<std::string, std::string>>
PronounsAlejoApi::pronounsFromId = {};

/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonObject object)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'parse' can be made static [readability-convert-member-functions-to-static]

Suggested change
/*static*/ UserPronouns PronounsAlejoApi::parse(QJsonObject object)
/*static*/ static UserPronouns PronounsAlejoApi::parse(QJsonObject object)

else
{
return {};
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: do not use 'else' after 'return' [llvm-else-after-return]

Suggested change
}
return {};

NetworkRequest(PronounsAlejoApi::API_URL + PronounsAlejoApi::API_USERS +
"/" + user.username)
.concurrent()
.onSuccess([request, user](auto result) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: no viable conversion from '(lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24)' to 'NetworkSuccessCallback' (aka 'function<void (NetworkResult)>') [clang-diagnostic-error]

            .onSuccess([request, user](auto result) {
                       ^
Additional context

/usr/include/c++/13/bits/std_function.h:374: candidate constructor not viable: no known conversion from '(lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24)' to 'nullptr_t' (aka 'std::nullptr_t') for 1st argument

      function(nullptr_t) noexcept
      ^

/usr/include/c++/13/bits/std_function.h:385: candidate constructor not viable: no known conversion from '(lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24)' to 'const function<void (NetworkResult)> &' for 1st argument

      function(const function& __x)
      ^

/usr/include/c++/13/bits/std_function.h:403: candidate constructor not viable: no known conversion from '(lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24)' to 'function<void (NetworkResult)> &&' for 1st argument

      function(function&& __x) noexcept
      ^

/usr/include/c++/13/bits/std_function.h:434: candidate template ignored: requirement '_Callable<(lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24), (lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24), std::__invoke_result<(lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24) &, chatterino::NetworkResult>>::value' was not satisfied [with _Functor = (lambda at /github/workspace/src/providers/pronouns/alejo/PronounsAlejoApi.cpp:134:24)]

	function(_Functor&& __f)
 ^

src/common/network/NetworkRequest.hpp:46: passing argument to parameter 'cb' here

    NetworkRequest onSuccess(NetworkSuccessCallback cb) &&;
                                                    ^

@DeltaTimo
Copy link
Author

I don't think I'll be able to update the PR this weekend, but I've installed the clang LS extension and I starting working through the suggestions yesterday.

The owner of PronounDB also replied yesterday I'll read it carefully tomorrow or so and share her comments.

Apologies for accidentally closing the PR yesterday, I was scrolling through the comments and somehow clicked "Close" 😒

@pajlada
Copy link
Member

pajlada commented Jun 8, 2024

Alright, I'll wait with reviewing the PR until you've had a chance to look at it further then. Poke me!

@treuks
Copy link
Contributor

treuks commented Jun 12, 2024

My recommendation is to only display the pronouns in the usercard - this minimizes http requests and avoids implementation complexity (of batching requests or caching thousands of pronouns)

There are certain pronoundb plugins for Discord, which show pronouns for a user. They show the pronouns inline with the messages, but only after you click on their profile, like so:

User message before you open the profile:
image

User message after you open the profile:
image

In my opinion this is a fairly decent approach to this sort of situation, we could do it like so:

Before:
image 1

Open user card:
Frame 1(12)

After:
image 4

I feel like this could be a decent middleground between only showing it in the usercard and making requests for like every user. thoughts?

@pajlada
Copy link
Member

pajlada commented Jun 12, 2024

I would be fine with that compromise @treuks

@iProdigy
Copy link
Contributor

compromise is good; i'm not a maintainer but could be easier to review if 2 separate prs

@Nerixyz
Copy link
Contributor

Nerixyz commented Aug 3, 2024

Thanks! I already started working on the clang-tidy bot comments as they were posted and I'll work on your comments probably some time next week when I find some time to. For some reason, I can't get clang-tidy to work, but clang-format should run on each save, but perhaps there weren't any changes in the files since I enabled it.

If you use clangd, you get clang-tidy "for free" (it runs as part of clangd). There are plugins for most editors (e.g. VS Code). If you don't want to use clangd, you can invoke clang-tidy like this: clang-tidy src/path/to/file.cpp -p build from the directory of the repository (build is a path to the CMake build folder; make sure you configured the project with -DCMAKE_EXPORT_COMPILE_COMMANDS=On).

@iProdigy
Copy link
Contributor

@DeltaTimo if you're short on time i'd be happy to work on the review comments to get this past the finish line if you want to add me as a collaborator

@DeltaTimo
Copy link
Author

DeltaTimo commented Aug 28, 2024

@DeltaTimo if you're short on time i'd be happy to work on the review comments to get this past the finish line if you want to add me as a collaborator

@iProdigy I have some changes ready to push, if I've missed anything feel free to work on it!

@Nerixyz Do I force push the changes on my branch?

@Nerixyz
Copy link
Contributor

Nerixyz commented Aug 28, 2024

Do I force push the changes on my branch?

Since there are review comments already, it's better not to (unless you substantially changed the design). In the end, they will be squashed anyway.

Copy link
Contributor

@Nerixyz Nerixyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much better! Only a few nitpicks and some of the old comments. I marked all comments that you addressed and all that are invalid (due to clang-tidy not knowing about the header) with 👎. I can't resolve them, but you can (by clicking on the "Resolve" button). This makes the https://github.com/Chatterino/chatterino2/pull/5442/files view a bit cleaner. I marked suggestions that are still "valid" with 👍 (really only nitpicks - looks good otherwise).

CHANGELOG.md Outdated
@@ -1,5 +1,9 @@
# Changelog

## Unreleased

- Minor: Add option to show pronouns in user card. (#5442)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved to Unversioned - and maybe a major entry(?).

src/Application.cpp Show resolved Hide resolved
@@ -147,6 +148,7 @@ Application::Application(Settings &_settings, const Paths &paths,
, logging(new Logging(_settings))
, linkResolver(new LinkResolver)
, streamerMode(new StreamerMode)
, pronouns(new pronouns::Pronouns)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: use std::make_shared<...> here (it creates a single allocation at the cost of weak pointers keeping the object alive). Also makes it more clear that this is a shared_ptr instead of a unique_ptr.

{
// Only fetch pronouns if we haven't fetched before.
{
std::shared_lock<std::shared_mutex> lock(this->mutex);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The template argument could be removed, as it is automatically deduced (same for the other locations).

Comment on lines 14 to 19
private:
// mutex for editing the saved map.
std::shared_mutex mutex;
// Login name -> Pronouns
std::unordered_map<QString, UserPronouns> saved;
AlejoApi alejoApi;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved below the public items (so that public items come first).

public:
Pronouns() = default;

void fetch(const QString &user,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
void fetch(const QString &user,
void fetch(const QString &username,

Signifying this isn't a userID (it's already renamed in the implementation)

qCDebug(chatterinoPronouns)
<< "Fetching pronouns from alejo.io for " << username;

alejoApi.fetch(username, [this, callbackSuccess, callbackFail,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I forgot this: As part of the "cleanly exit Chatterino" effort: This shouldn't capture this, but a weak_ptr to this (obtained through weak_from_this; and the class has to inherit from std::enable_shared_from_this).

Similarly, for AlejoApi, which shouldn't capture this in network requests. It could store a std::weak_ptr<void> to its parent (Pronouns) on construction and use that to ensure it's still alive when receiving a response for network requests.

But that isn't something you have to address - I suspect pajlada will take care of that before merging.

// mutex for editing the saved map.
std::shared_mutex mutex;
// Login name -> Pronouns
std::unordered_map<QString, UserPronouns> saved;
Copy link
Contributor

@iProdigy iProdigy Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will work for now - in the future if we start doing bulk lookups, we'll want to switch to lru_cache to avoid a (bigger) memory leak

@DeltaTimo
Copy link
Author

Looks much better! Only a few nitpicks and some of the old comments. I marked all comments that you addressed and all that are invalid (due to clang-tidy not knowing about the header) with 👎. I can't resolve them, but you can (by clicking on the "Resolve" button). This makes the https://github.com/Chatterino/chatterino2/pull/5442/files view a bit cleaner. I marked suggestions that are still "valid" with 👍 (really only nitpicks - looks good otherwise).

Ah I was wondering where the 👎 comments were but Felanbird has apparently resolved them already.

I'll work on the rest real quick!

Copy link
Contributor

@Nerixyz Nerixyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some tiny things 👍

src/providers/pronouns/UserPronouns.cpp Outdated Show resolved Hide resolved
src/providers/pronouns/UserPronouns.hpp Outdated Show resolved Hide resolved
src/providers/pronouns/UserPronouns.hpp Outdated Show resolved Hide resolved
src/providers/pronouns/alejo/PronounsAlejoApi.hpp Outdated Show resolved Hide resolved
src/providers/pronouns/alejo/PronounsAlejoApi.hpp Outdated Show resolved Hide resolved
Copy link
Contributor

@Nerixyz Nerixyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍 If you want, you can try to merge with master (not rebase). Other than that, your PR is still marked as a draft - not sure how CI is configured, but it should run when you mark it as reviewable.

@Wissididom
Copy link
Contributor

Wissididom commented Aug 29, 2024

Afaik the reason for the CI not running are the conflicts

@DeltaTimo
Copy link
Author

Awesome! I'll try to merge and remove the draft tag.

@DeltaTimo DeltaTimo changed the title feat: Fetch and show pronouns for chatters. Fetch and show pronouns for chatters. Aug 29, 2024
@DeltaTimo DeltaTimo marked this pull request as ready for review August 29, 2024 15:04
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@@ -124,6 +129,7 @@ class MockApplication : public mock::BaseApplication
FfzEmotes ffzEmotes;
SeventvEmotes seventvEmotes;
DisabledStreamerMode streamerMode;
Pronouns pronouns;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: field has incomplete type 'chatterino::pronouns::Pronouns' [clang-diagnostic-error]

    Pronouns pronouns;
             ^
Additional context

src/Application.hpp:56: forward declaration of 'chatterino::pronouns::Pronouns'

    class Pronouns;
          ^

@@ -124,6 +129,7 @@
FfzEmotes ffzEmotes;
SeventvEmotes seventvEmotes;
DisabledStreamerMode streamerMode;
Pronouns pronouns;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: unknown type name 'Pronouns'; did you mean 'chatterino::pronouns::Pronouns'? [clang-diagnostic-error]

Suggested change
Pronouns pronouns;
chatterino::pronouns::Pronouns pronouns;
Additional context

src/Application.hpp:56: 'chatterino::pronouns::Pronouns' declared here

    class Pronouns;
          ^

@@ -257,6 +257,13 @@ class EmptyApplication : public IApplication
return nullptr;
}

pronouns::Pronouns *getPronouns() override
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'getPronouns' can be made static [readability-convert-member-functions-to-static]

Suggested change
pronouns::Pronouns *getPronouns() override
static pronouns::Pronouns *getPronouns() override

@@ -257,6 +257,13 @@
return nullptr;
}

pronouns::Pronouns *getPronouns() override
{
assert(false && "EmptyApplication::getPronouns was called without "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: implicit conversion 'const char *' -> 'bool' [readability-implicit-bool-conversion]

        assert(false && "EmptyApplication::getPronouns was called without "
                        ^

this fix will not be applied because it overlaps with another fix

pronouns::Pronouns *getPronouns() override
{
assert(false && "EmptyApplication::getPronouns was called without "
"being initialized");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: redundant boolean literal supplied to boolean operator [readability-simplify-boolean-expr]

Suggested change
"being initialized");
assert(false);

AccountController accounts;
mock::MockTwitchIrcServer twitch;
Emotes emotes;
BttvEmotes bttvEmotes;
FfzEmotes ffzEmotes;
SeventvEmotes seventvEmotes;
Pronouns pronouns;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: field has incomplete type 'chatterino::pronouns::Pronouns' [clang-diagnostic-error]

    Pronouns pronouns;
             ^
Additional context

src/Application.hpp:56: forward declaration of 'chatterino::pronouns::Pronouns'

    class Pronouns;
          ^

AccountController accounts;
mock::MockTwitchIrcServer twitch;
Emotes emotes;
BttvEmotes bttvEmotes;
FfzEmotes ffzEmotes;
SeventvEmotes seventvEmotes;
Pronouns pronouns;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: unknown type name 'Pronouns'; did you mean 'chatterino::pronouns::Pronouns'? [clang-diagnostic-error]

Suggested change
Pronouns pronouns;
chatterino::pronouns::Pronouns pronouns;
Additional context

src/Application.hpp:56: 'chatterino::pronouns::Pronouns' declared here

    class Pronouns;
          ^

@@ -91,6 +91,20 @@ class MockApplication : public mock::BaseApplication
return &this->seventvEmotes;
}

<<<<<<< HEAD:tests/src/TwitchMessageBuilder.cpp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: version control conflict marker in file [clang-diagnostic-error]

<<<<<<< HEAD:tests/src/TwitchMessageBuilder.cpp
^

@@ -102,6 +116,7 @@
BttvEmotes bttvEmotes;
FfzEmotes ffzEmotes;
SeventvEmotes seventvEmotes;
Pronouns::Pronouns pronouns;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: field has incomplete type 'pronouns::Pronouns' [clang-diagnostic-error]

    Pronouns::Pronouns pronouns;
                       ^
Additional context

src/Application.hpp:56: forward declaration of 'chatterino::pronouns::Pronouns'

    class Pronouns;
          ^

@@ -102,6 +116,7 @@
BttvEmotes bttvEmotes;
FfzEmotes ffzEmotes;
SeventvEmotes seventvEmotes;
Pronouns::Pronouns pronouns;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: use of undeclared identifier 'Pronouns'; did you mean 'pronouns'? [clang-diagnostic-error]

Suggested change
Pronouns::Pronouns pronouns;
pronouns::Pronouns pronouns;
Additional context

src/Application.hpp:55: 'pronouns' declared here

namespace pronouns {
          ^

qCDebug(chatterinoPronouns) << "Fetching available pronouns for alejo.io";
NetworkRequest(AlejoApi::API_URL + AlejoApi::API_PRONOUNS)
.concurrent()
.cache()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How well or not well does this handle caching?
From my cursory glance at the caching system, I can see a scenario where:

  • the data gets loaded and saved to cache
  • a user adds themselves to the database
  • this gets loaded from cache but still has the old data.

Even restarting Chatterino probably wouldn't help because the cached data will still be in cache and it will just be reloaded.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only for the pronouns (i.e. some-pronoun-id → "they/them") not for the users, thus it's not frequently updated. But I agree that it should be removed, because we don't have any way of invalidating this.

Comment on lines 94 to 107
<<<<<<< HEAD:tests/src/TwitchMessageBuilder.cpp
IStreamerMode *getStreamerMode() override
{
return &this->streamerMode;
}

pronouns::Pronouns *getPronouns() override
{
return &this->pronouns;
}

Settings settings;
=======
>>>>>>> master:tests/src/MessageBuilder.cpp
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still have merge conflict markers here

Suggested change
<<<<<<< HEAD:tests/src/TwitchMessageBuilder.cpp
IStreamerMode *getStreamerMode() override
{
return &this->streamerMode;
}
pronouns::Pronouns *getPronouns() override
{
return &this->pronouns;
}
Settings settings;
=======
>>>>>>> master:tests/src/MessageBuilder.cpp
IStreamerMode *getStreamerMode() override
{
return &this->streamerMode;
}
pronouns::Pronouns *getPronouns() override
{
return &this->pronouns;
}
Settings settings;

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, that's odd. git diff --check didn't catch that (or perhaps more likely, I forgot to run it one last time)

@DeltaTimo
Copy link
Author

How can I run the benchmarks, tests, etc. on my machine? I'm getting Unknown CMake command "add_sanitizers".

In any case, I've pushed some changes that should hopefully fix the build errors.

@Nerixyz
Copy link
Contributor

Nerixyz commented Aug 29, 2024

I'm getting Unknown CMake command "add_sanitizers".

Are you sure you properly checked out all submodules (git submodule update --init --recursive)?

@DeltaTimo
Copy link
Author

DeltaTimo commented Aug 29, 2024

I'm getting Unknown CMake command "add_sanitizers".

Are you sure you properly checked out all submodules (git submodule update --init --recursive)?

I didn't, that was definitely it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants