RSS Feeds stopped working a month ago

[tio-trom]

My RSS feeds, all of them, stopped working some 2 weeks ago. I checked a bunch of them, they all work outside Friendica. My instance version is 2022.06. I wonder what I can check or do. I have a lot of these:

[annando]

Are these feed contacts under your own user or have you setup own users that publish the feeds?

[tio-trom]

These are external RSS feeds that I have added as my contacts. They look like this https://ytb.trom.tf/feed/channel/UCcvfHa-GHSOHFAjU0-Ie57A or like this https://teddit.net/r/popular/top?api&type=rss

[tio-trom]

For example for the reddit popular last update was on 1st of september, more than 3 weeks ago:

But if you check that RSS URL https://teddit.net/r/popular/top?api&type=rss it is constantly updated. I tried to "Update now" via Friendica's Actions menu, but nothing happens.

[MrPetovan]

Anecdata: I'm still receiving RSS items in my Network page as recently as 16 hours ago.

[tio-trom]

Anecdata: I'm still receiving RSS items in my Network page as recently as 16 hours ago.

I envy you :D - I used my friendica as my RSS reader too, and now nothing works :( and I did nothing. No updates....I even rebooted the servers a few times just in case.

[SpcCw]

I have several separate RSS accounts which have RSS feeds added to their contacts and mirror the posts.

A month ago or so they stopped working while RSS sources were still available. Yesterday I went to one of them and re-added RSS feed - then magically it started working again.

Not sure if it is permanent fix but seems to work for now.

Also make sure RSS feeds are available from your Friendica server (curl it from there or something) - once an RSS source banned my Friendica for some reason, too many requests probably.

[tio-trom]

Ohhh....so there might be a bug somewhere then. Because that's when it started for me too. And I use plenty of RSS feeds from plenty of sources so for sure there is no ban. Would be a pain to re-add them all one by one.....maybe someone can suggest some tests we may do to uncover where the issue is.

[tio-trom]

So I checked multiple RSS feeds they all stopped working a month or so ago. I think it was around the time when I updated my server to Debian 11. But I mean it is strange that just that stopped working and now the rest - like other federated accounts work well. The strange bit is that Twitter stopped working too. I was getting updates from a bunch of Twitter users via the Twitter addon. The Twitter side is all fine app-access wise. How can these be related? What could I try to do?

@SpcCw are you using Yunohost? Or you use my instance social.trom.tf? I want to figure if this is limited to my instance...

[annando]

Does the worker run at all?

[SpcCw]

@tio-trom - no, I use some other VPS and installed Friendica manually few years ago, since then it runs mostly develop branch.

RSS worked great until month or two ago, I had many separate RSS accounts so other people could subscribe to them. Then all of them failed. A few days ago I just re-added RSS contact to one of bots and it seems to work again. I am certain it was introduced with some Friendica update (since I did not update anything else) but not sure which one.

[tio-trom]

Does the worker run at all?

Am unsure how to check that or what you mean. My instance works perfectly fine, except for the RSS and Twitter. I even added new RSS feeds and do not see them at all.

[annando]

What happens when you call the worker by hand in the command line? Does the execution take some time? Do you receive any messages?

[tio-trom]

I do not know how to do that. But based on this https://github.com/friendica/friendica/blob/2022.09-rc/doc/Install.md#required-background-tasks I went to the friendica folder and then did either bin/worker.php or /usr/bin/php bin/worker.php - for both there is no output. It "hands".

[tio-trom]

@SpcCw we had an interesting discussion over the matrix chat about this. Here is the chat url https://matrix.to/#/#friendi.ca:matrix.org - basically @annando said that likely my system is busy with other tasks and forgets to pull the content from RSS. Is your server busy too? Maybe we can find a smoking gun.

[tio-trom]

Ok works now. I am not sure what fixed it, but from what I remember talking to Michael and what I've done:

I removed some parameters for Friendica such as

update workerqueue set done = 1 where command = 'FetchFeaturedPosts';
update workerqueue set done = 1 where command = 'UpdateContact';

That killed some 80.000 processes that were stuck in the queue.

Then I tweaked some files. Mind you you may have these somewhere else. I use Yunohost. In /etc/mysql/mariadb.conf.d/50-server.cnf I added:

[mariadb]
key_buffer_size = 16K
max_allowed_packet = 16M
table_open_cache = 4
sort_buffer_size = 4M
read_buffer_size = 256K
read_rnd_buffer_size = 256K
net_buffer_length = 2K
thread_stack = 128K
table_definition_cache = 500
innodb_buffer_pool_size = 30G

In /etc/php/7.4/fpm/pool.d/friendica.conf I made sure to have these:

pm = dynamic
pm.max_children = 100
pm.start_servers = 10

pm.min_spare_servers = 10

pm.max_spare_servers = 30

pm.max_requests = 500
    
    
php_admin_value[upload_max_filesize] = 50M
php_admin_value[post_max_size] = 500M
php_admin_value[memory_limit] = 512M

Then in /etc/nginx/conf.d/social.trom.tf.d/friendica.conf to have these

#Execute and serve PHP files
    location ~* \.php$ {
        fastcgi_buffers 8 16k;
        fastcgi_buffer_size 32k;
        fastcgi_connect_timeout 60;
        fastcgi_send_timeout 300;
        fastcgi_read_timeout 300;
        fastcgi_split_path_info ^(.+?\.php)(/.*)$;
        try_files $uri =404;
        fastcgi_pass unix:/var/run/php/php7.4-fpm-friendica.sock;
        fastcgi_index index.php;	
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }

Of course restart the database, restart PHPfpm. I can keep this one open till @SpcCw fixes it for him too.

I am not sure if Twitter works tho....but this was mostly about the RSS feeds.

[tio-trom]

Ok Twitter also works. Hurray!!!

[MrPetovan]

So it was just a slow worker?

[tio-trom]

So it was just a slow worker?

Apparently.... Michael helped me a lot on the Matrix Friendica chat. My server was like "oh we have to also pull some RSS but first let me do these TENS of THOUSANDS of other requests at a very slow pace..." - and it was unable to do them in time, to then pull the RSS contents....that's what we think happened.

[SpcCw]

@tio-trom - I think it can be closed as it seems to work for me too and I probably had different problem.

Thanks for details on the solution! :)