Blocked domains flood gserver entries #12700

Quix0r · 2023-01-20T02:03:06Z

It has shown in the past that trolls from the infamous gab.best and also more recently activitypub-troll.cf domains have used a flaw in Friendica to flood the gserver table with millions (see this link) of useless records (tons of invalid and not findable sub domains) of records that just pile up and seem to never end.

With this change, I was finally able to block them from entering my and other's database tables. This PR also contains a change to to avoid redundant invocations of Strings::normaliseLink() which can safe some CPU cycles and speedup the code.

- some trolls managed to flood gserver with useless URLs. They can be blocked by domain blocking them, but still it floods gserver table with dead entries - this hack tries to change that so they won't enter gserver at all. Let's hope these trolls as `activitypub-trolls.cf` learn a lesson (and get adults soon) Signed-off-by: Roland Häder <roland@mxchange.org>

- just for consistency ...

src/Model/GServer.php

MrPetovan · 2023-01-20T03:49:14Z

src/Model/GServer.php

+		} elseif (Network::isUrlBlocked($url)) {
+			Logger::info('Server domain is blocked', ['url' => $url]);
+			return false;


elseif here is superfluous since there's a return in the parent condition.

Suggested change

} elseif (Network::isUrlBlocked($url)) {

Logger::info('Server domain is blocked', ['url' => $url]);

return false;

}

if (Network::isUrlBlocked($url)) {

Logger::info('Server domain is blocked', ['url' => $url]);

return false;

What do you mean by "parent condition"?

The condition where the elseif originated.

I usually only make a 2nd if() block if there is code between the upper and the then added lower one:

if (someCondition) { // Do something, like logging, closing/freeing resources return false; } // Do other things: doOtherCode(); if (otherCondition) { // Do something, like logging, closing/freeing resources return false; }

Otherwise, when there is no code in between these two if() blocks, I merge them with elseif() or when they do the exact same thing, I merge them with ||.

Let me provide a more actionable suggestion.

Suggested change

} elseif (Network::isUrlBlocked($url)) {

Logger::info('Server domain is blocked', ['url' => $url]);

return false;

if (!Strings::compareLink($url, $original_url)) {

self::setFailureByUrl($original_url);

if (!self::getID($url, true)) {

self::detect($url, $network, $only_nodeinfo);

}

return false;

}

if (Network::isUrlBlocked($url)) {

Logger::info('Server domain is blocked', ['url' => $url]);

return false;

}

These condition have nothing to do with each other because of the return false. If there was anything but a return a elseif would be warranted, but not in this case where it adds complexity for no benefit.

src/Worker/UpdateServerPeers.php

Quix0r · 2023-01-20T09:26:18Z

DELETE FROM `gserver` WHERE `url` LIKE '%activitypub-troll.cf%' OR `url` LIKE '%gab.best%'

Query executed OK, 0 rows affected. (0.149 s)

So far really effective! Without this change, both the later one would have flooded again with even millions(!) of sub domains.

- moved if() block to suggested position by MrPetovan, for me I want to have all conditions checked at the start of the method, e.g. no unwanted null references or (in this case) if the URL is blacklisted - normalized URLs are without SSL, means http://host/path/file.ext so they exist only once for contacts and servers (aka. instances) - documented returned type `void`

src/Model/GServer.php

annando · 2023-01-22T06:50:06Z

src/Model/GServer.php

 			Logger::info('Set failed status for existing server', ['url' => $url]);
 			if (self::isDefunct($gserver)) {
 				self::archiveContacts($gserver['id']);
 			}
 			return;
 		}
-		self::insert(['url' => $url, 'nurl' => Strings::normaliseLink($url),
+
+		if (Network::isUrlBlocked($url)) {


This block here is not needed. AFAIK we don't call this function at all for blocked servers. If we should, then we had to check earlier for blocked servers and should call the appropriate function from there.

src/Model/GServer.php

src/Worker/UpdateServerPeers.php

tek-aevl · 2023-04-04T10:13:24Z

This is killing my power bill, I have 1000 of duplicate entry errors, because of this same issue.

MrPetovan · 2023-04-04T11:17:17Z

Can you please share a sample of duplicate entry errors?

tek-aevl · 2023-04-13T07:56:57Z

Ok, I forgot to respond to this sooner,

Event details

Date	Level	Context	Message
2023-04-12T12:50:56Z	ERROR	worker	DB Error

Data

code

error

MySQL server has gone away

callstack

GServer::insert, GServer::setFailure, GServer::detect, GServer::check, GServer::getID, GServer::add, UpdateServerPeers::execute, Worker::execFunction

params

INSERT INTO `gserver` (`url`, `nurl`, `network`, `created`, `last_failure`, `failed`) VALUES ('https://32tbqwum0.activitypub-troll.cf', 'http://32tbqwum0.activitypub-troll.cf', 'unkn', '2023-04-12 12:50:56', '2023-04-12 12:50:56', 1)

2023-04-09T15:03:00Z	ERROR	worker	DB Error

Data

code

error

Duplicate entry 'http://39cqpqb9m.activitypub-troll.cf' for key 'nurl'

callstack

GServer::insert, GServer::setFailure, GServer::detect, GServer::check, GServer::getID, GServer::add, UpdateServerPeers::execute, Worker::execFunction

params

INSERT INTO `gserver` (`url`, `nurl`, `network`, `created`, `last_failure`, `failed`) VALUES ('https://39cqpqb9m.activitypub-troll.cf', 'http://39cqpqb9m.activitypub-troll.cf', 'unkn', '2023-04-09 15:03:00', '2023-04-09 15:03:00', 1)

MrPetovan · 2023-04-13T13:20:53Z

Can you please confirm what version of Friendica you are running and what pattern you set in your server block list related to this attack?

tek-aevl · 2023-04-16T01:06:11Z

https://www.urbanmind.net/friendica

Using PiHole blacklisting, with (.|^)activitypub-troll.cf$ and cloudflare dns firewall.

MrPetovan · 2023-04-16T11:33:09Z

Ok, the *.activitypub-troll.cf pattern looks good and you seem to be running the 2023.01 version. I also see you have enabled some sort of CloudFlare DDoS protection which prevents JS and CSS files to load.

MrPetovan · 2023-04-16T11:56:40Z

So, I have looked into the chain of calls you provided, and it should have been stopped in 4 different places given the error and your blocked server domain pattern:

In src/Worker/UpdateServerPeers.php:60 when we check for the existence of a GServer by normalized URL.
In src/Model/GServer.php:127 when we try to retrieve a GServer by normalized URL.
In src/Model/GServer.php:344 when we try to retrieve a GServer by normalized URL.
In src/Model/GServer.php:412 when we try to retrieve a GServer by normalized URL.

I don't have a good explanation for this.

I also note that we've added a number of checks against the server domain pattern block list in the upcoming version 2023.04 so updating to it should reduce your woes.

tek-aevl · 2023-04-17T07:45:17Z

The CSS not loading, was not intended, but it helps.

I think I fixed the css not loading issue, it was hidden in my web shield settings.

Quix0r added 2 commits January 20, 2023 01:50

Continued:

ba08692

- just for consistency ...

MrPetovan requested changes Jan 20, 2023

View reviewed changes

MrPetovan mentioned this pull request Jan 20, 2023

Been spammed by *.activitypub-troll.cf, how to delete related items in storage and database #12705

Closed

MrPetovan reviewed Jan 21, 2023

View reviewed changes

src/Model/GServer.php Show resolved Hide resolved

MrPetovan approved these changes Jan 21, 2023

View reviewed changes

MrPetovan merged commit 0681f94 into friendica:develop Jan 21, 2023

MrPetovan added this to the 2023.03 milestone Jan 21, 2023

MrPetovan added the Enhancement label Jan 21, 2023

annando reviewed Jan 22, 2023

View reviewed changes

src/Model/GServer.php Show resolved Hide resolved

annando reviewed Jan 22, 2023

View reviewed changes

src/Worker/UpdateServerPeers.php Show resolved Hide resolved

Quix0r deleted the features/blocklist-gserver branch January 27, 2023 08:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Blocked domains flood gserver entries #12700

Blocked domains flood gserver entries #12700

Quix0r commented Jan 20, 2023 •

edited

MrPetovan Jan 20, 2023

Quix0r Jan 20, 2023

MrPetovan Jan 20, 2023

Quix0r Jan 21, 2023

MrPetovan Jan 21, 2023 •

edited

Quix0r commented Jan 20, 2023

annando Jan 22, 2023

tek-aevl commented Apr 4, 2023

MrPetovan commented Apr 4, 2023

tek-aevl commented Apr 13, 2023 •

edited by MrPetovan

Event details

Data

Data

MrPetovan commented Apr 13, 2023

tek-aevl commented Apr 16, 2023 •

edited

MrPetovan commented Apr 16, 2023 •

edited

MrPetovan commented Apr 16, 2023

tek-aevl commented Apr 17, 2023 •

edited

Blocked domains flood gserver entries #12700

Blocked domains flood gserver entries #12700

Conversation

Quix0r commented Jan 20, 2023 • edited

MrPetovan Jan 20, 2023

Choose a reason for hiding this comment

Quix0r Jan 20, 2023

Choose a reason for hiding this comment

MrPetovan Jan 20, 2023

Choose a reason for hiding this comment

Quix0r Jan 21, 2023

Choose a reason for hiding this comment

MrPetovan Jan 21, 2023 • edited

Choose a reason for hiding this comment

Quix0r commented Jan 20, 2023

annando Jan 22, 2023

Choose a reason for hiding this comment

tek-aevl commented Apr 4, 2023

MrPetovan commented Apr 4, 2023

tek-aevl commented Apr 13, 2023 • edited by MrPetovan

Event details

Data

Data

MrPetovan commented Apr 13, 2023

tek-aevl commented Apr 16, 2023 • edited

MrPetovan commented Apr 16, 2023 • edited

MrPetovan commented Apr 16, 2023

tek-aevl commented Apr 17, 2023 • edited

Quix0r commented Jan 20, 2023 •

edited

MrPetovan Jan 21, 2023 •

edited

tek-aevl commented Apr 13, 2023 •

edited by MrPetovan

tek-aevl commented Apr 16, 2023 •

edited

MrPetovan commented Apr 16, 2023 •

edited

tek-aevl commented Apr 17, 2023 •

edited