Adjust rich_embeds filter so it doesn't catch edge case

[bast0006]

Right now, discord converts twitter links when applying the standard twitter embed treatment. Our check for rich embeds determines whether an embed is autogenerated or not by checking if the embed link is present within the source message here

bot/bot/exts/filters/filtering.py

Line 520 in fa02180

async def _has_rich_embed(msg: Message) -> Union[bool, List[discord.Embed]]:

, but this does not account for discords transformation of mobile.twitter.com links to twitter.com links, and thus a message with mobile.twitter.com in it is falsely detected as a rich embed for the watchlist.

This is probably fixable by detecting if the link goes to twitter.com and there is a link to mobile.twitter.com with the same parameters. That may get a little tricky, but twitter links aren't complex links so perhaps simple string operations will suffice.

We may also want to consider writing an "apply_link_transformations" function that will extract and transform relative links within a message to compare against the embed.

[Akarys42]

Superseded by #1530

[scragly]

Considering this being an ongoing issue, it's unnecessary to block this bugfix until the new system. This is now open to implementing a fix for by anyone interested in doing so.

[shtlrs]

Discussed in discord with @mbaruh.

I'll reproduce the issue, and when I understand that, I'll drop the proposition here and we can discuss it.

[shtlrs]

@bast0006 Can you please provide the examples you've tried ?
Because I'm not sure I understand correctly here since you said

and thus a message with mobile.twitter.com in it is falsely detected as a rich embed for the watchlist.

When I use this link https://twitter.com/Nadallica86/status/1574145915048398854 or https://mobile.twitter.com/nedbat/status/1504062673809338370, _has_rich_embed works just fine.

But as soon as I use twitter's Lite version (with the 'mobile' prefix), e.g. https://mobile.twitter.com/Nadallica86/status/1574145915048398854 or https://mobile.twitter.com/nedbat/status/1504062673809338370, the if msg.embeds: doesn't even run, because it's currently detected as a message with no embeds, when in fact it does.

[bast0006]

https://mobile.twitter.com/nedbat/status/1504062673809338370 is rendered as the following:

{'author': {'icon_url': 'https://pbs.twimg.com/profile_images/1559004270170628096/AaSLxb_F_400x400.jpg',
            'name': 'Ned Batchelder (@nedbat)',
            'proxy_icon_url': 'https://images-ext-1.discordapp.net/external/KcDnbosWojYPkPQ4Ummp1Ad-f2wJ8Ukq3PXnU7IK-ZU/https/pbs.twimg.com/profile_images/1559004270170628096/AaSLxb_F_400x400.jpg',
            'url': 'https://twitter.com/nedbat'},
 'color': 1942002,
 'description': 'A list in #python can contain itself as an element!',
 'fields': [{'inline': True, 'name': 'Likes', 'value': '686'}],
 'footer': {'icon_url': 'https://abs.twimg.com/icons/apple-touch-icon-192x192.png',
            'proxy_icon_url': 'snip',
            'text': 'Twitter'},
 'image': {'height': 452,
           'proxy_url': 'snip',
           'url': 'https://pbs.twimg.com/media/FN9skDtXoAMLWKI.jpg',
           'width': 912},
 'timestamp': '2022-03-16T11:51:00.226000+00:00',
 'type': 'rich',
 'url': 'https://twitter.com/nedbat/status/1504062673809338370'}

note how the url field here is for https://twitter.com/nedbat/status/1504062673809338370, but the url inside the message is https://mobile.twitter.com/nedbat/status/1504062673809338370, and the embed is of type rich.

I just tested with that link, and it does trigger the alert, as you can see: