Abuse mail received because of "unusual traffic" #8663
Assignees
Labels
Comments
|
I will also reply to the mail and ask them if they - by any chance - are running other federated software on that domain. |
|
At some time there seemed to had been some Fediverse server there. Otherwise there shouldn't be some discovery. Could you post something about this in the developers forum? I also added the domain to the blocklist now. |
|
I have posted it to the admins forum. And I asked them about the possibility that they are or were running some other federated network software. |
|
Btw. who is receiving the mails sent to |
|
Mails are sent to some persons, including me and Tobias. I haven't checked it though, but got a note from Tobias about this today while being at work. |
|
Hi, folks The only federated software we operate and are expecting to see traffic for is a Matrix chat instance |
|
@alyxw is your server on the-federation.info? We poll the list of servers from it to detect new systems. |
|
Yes, it looks like our server was added however neither I or my colleague added it |
|
I guess we need some kind of negative caching as well, so that we don't probe it as many times. |
|
No kidding, in some cases in the logs it looks like our server would return a 302 and the Friendica server would retry 10+ more times in a matter of seconds |
|
Sorry for that. We will work at this. |
|
If there's anything I can do to help, let me know. an example snippet with the quick retries |
|
I will have to do some tests from my server pirati.ca, so you will see several requests from there again (but not today, since it's getting late). I guess I will find some good way tomorrow. Then the system will still do the probing, but it will store the result so that it will test only some time later (with an increasing interval). The changes will be for the develop branch with the version |
|
You've been added on my block list as well. The patterns repeat for all Friendica servers (mine included) so I believe we can fix it. |
|
I added some negative caching on my server. I will watch the log file over the day. And when this looks well, I will create a PR this evening. At maximum this patch should reduce the connections to one per day. But I will have a look at the check interval as well, so that eventually the checks will be reduced to one check per week. Update: One thing that I still don't understand is the high frequency that single servers are performing these requests. I looked into my logfile - and my servers only tried once and gave up for some time. |
|
I created a first PR. A second one will follow later that introduces longer retrial pauses. But this is nothing that I want to code before work and breakfast. |
|
On Tue, 19 May 2020 12:41:23 -0700 Steffen K9 ***@***.***> wrote:
Btw. who is receiving the mails sent to `info(at)friendi.ca`? The
original mail was sent to that address.
This address is an mailing list and the mail should be distributed to the main persons of the development community. I've also made a posting about the problem on Friendica with some devs as audience.
|
I have the feeling it's a parallel process issue where multiple worker threads are probing the same remote server at the same time but the pattern is too neat to be just an unlucky coincidence. |
|
Moved to separate issue I just checked my log file to see what else we are doing. :-) I grep-ed for 'host-meta'. We are apparently requesting server endpoints on user profiles. For example 'https://some-server.com/users/some-user-name/.well-known/host-meta'. All of those requests are answered with a 404 error. And in some cases we are doing it excessively. This doesn't make any sense. Or am I wrong? Here is a "short" part of my log. Look at the time stamps. |
|
@AlfredSK can you check if the |
|
Moved to separate issue I already did that. No, it doesn't. I checked the gserver table for The example in my above comment is an extreme one. I don't see that excessive behaviour very often. But we are requesting those URLs a lot. As far as I could see only Pleroma and Mastodon nodes are involved. But this could be due to the large share of these servers in the total number of servers. |
|
Moved to separate issue |
|
Moved to separate issue Examples: |
|
Moved to separate issue The worker is also checking these URLs for 'UpdateContact': |
|
Can we move this to a separate issue? I guess this is a complete different issue with a complete different solution. |
Done. #8675 |
|
I am seeing the same on my matrix server (synapse) |
|
We applied changes to the version 2020.06. On a first glance I don't see any requests from this version in your log, only prior versions. I hope that the changes we did to the current develop branch already reduced the load? |
|
its not really a load problem. I just find it weird that friendica tries to talk to matrix servers, even thought it doesn't seem to implement the matrix spec (correct me if i'm wrong), on endpoints that are not part of the matrix spec. Wouldn't it make sense to filter the the-federation.info list by protocol? I'm not sure if |
|
Yeah, that's the same. Okay, I haven't seen that request. For technical reasons we cannot completely set the requests to zero. I guess that your host is in the list of But the number of requests should be reduced a lot with the new code. |
|
Yes my server is on
|
|
The problem is that the only machine readable endpoint doesn't :-( |
|
graphql is designed to be machine readable and additionally has a fancy gui to build and test out queries. curl -v 'https://the-federation.info/graphql?query=query%20Nodes%20%7B%0A%20%20nodes(protocol%3A%20%22activitypub%22)%20%7B%0A%20%20%20%20host%0A%20%20%7D%0A%7D%0A&operationName=Nodes&variables=' |
|
Okay, I have to check that. |
|
@tilosp I was able boiling the query down to this: But the question remains how to perform "or" requests, since we support multiple protocols. I had a peak into the documentation, but haven't found something helpful here. |
|
I don't think that currently possible, but But currently it is still possible to get the lists for multiple protocols in one request. For example for ostatus and activitypub. The only disadvantage of this approach is that it contains duplicated hosts if one host supports multiple protocols. I am not sure if that is a problem since |
|
Ooooh, do individual commits have to mention |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Actual behavior
I have received a friendly abuse mail from the admin of
en0.io. They provided a log file with a lot of requests from Friendica nodes.As I see in that log file the requests are some probing stuff for well-known endpoints. But they say that they don't run a Friendica node and also never were running one.
Examples from the provided log:
The log file is pretty big. So this is only an example. I see probably all known Frindica nodes in that log file.
They are asking for reducing or eliminating the traffic.
I have added the domain to my server blocklist on all nodes under my control.
I will also forward this to the Friendica Admins Forum.
Friendica version you encountered the problem
Probably all of them. 2020.03 and 2020.06-dev are running on my domains.
The text was updated successfully, but these errors were encountered: