Need help for changing the feed sort order from lastSeen to the publish date #2083

hanan-cohen · 2018-10-27T11:00:37Z

Hello,

I am deploying a public feed aggregator and I will frequently add new feeds. The current sorting of the feed by lastSeen will look strange to the visitors.

I would like to change the sort order of the feed to be by the publishing date of the post (date) instead of by the date it was seen by the aggregator (lastSeen).

I searched the code and didn't find where to do that. Help will be appreciated.

Thanks.

Alkarex · 2018-10-27T13:57:42Z

Hello,
This is a topic that has been often times addressed, e.g. #1771 (comment)
The short version is:

If you let run FreshRSS for a few days with a refresh cron job (e.g. every 30min), the sorting will look better
Advertised date is not reliable enough to be useful, and using it creates many problems
Most other systems (including larger news aggregators but also e-mail clients etc.) are all using the same approach
Visitors are accustomed to items not strictly sorted by their declared / publication date, as it is also the case on popular systems such as Facebook, LinkedIn, Twitter, etc.
For our APIs (mobile clients) and for paging during scrolling, the guaranties given by the fact that no out-of-sequence article will be inserted allow for some important optimisations.

What we could easily do is to make an option to replace in the UI the advertised date by the discovery date, but I am not so found of the idea.

Feel free to post a link and/or screenshot, so that we can have a better idea of your precise situation, which depends on the feeds you are following, whether they support push (PubSubHubbub / WebSub) and if not, at which frequency you retrieve them.

Alkarex · 2018-10-27T14:00:32Z

P.S. the articles are sorted by the date they were first added to the database, not by the last seen date (which would not make sense).

hanan-cohen · 2018-10-27T15:55:51Z

Thanks.

I have read #1771 and this is the reason I explained why I need the change.

If it was only for my personal use, that would be OK. Since it's a public website, the users will be confused by the sorting order when I add a new blog.

The live site is at https://blogim.info/

It's a mix of Enslish/LTR and Hebrew/RTL. Work in progress.

As for lastSeen, this is the name of the value in the DB for "date added". I will loot at the code at try to figure out how to sort by date (publish date).

Alkarex · 2018-10-27T16:21:31Z

No, lastSeen is the last seen date (at its name implies), not the date when the article was added.
Changing the SQL requests to modify the sort order is trivial, but that will break features such as scrolling / paging, mark all as read, etc.

I invite you to check the other systems that your users might be familiar with (e-mails, social media platforms, other RSS aggregators, Planet systems, social links...). E-mail clients (Outlook, GMail...) are using received date. Other RSS aggregators (including popular - current or past - ones such as Google Reader, Digg, Feedly, etc.), social media are using a multi-criteria ranking, so users are rarely exposed to a strict sort by publication date.

You can simply hide this publication date if it is a problem for you:

Or show the received date instead (like in an e-mail client) with a simple change in the code (we might make that an option).

We have a work-in-progress Hebrew translation, in which contributions would be welcomed https://github.com/FreshRSS/FreshRSS/tree/dev/app/i18n/he

tetsuhou · 2020-12-02T16:03:38Z

Hello, I also would like to be able to sort articles by the publishing date, but it seems difficult.

Would it be possible to add an option that allows me to add a new feed without getting the current item, but just the new article?

ghost · 2022-04-21T01:07:34Z

Since this issue is still open, I'd like to add my support for this.

Sorting by date fetched tends to cause articles from the same source to get grouped together instead of having organic variety in feed content, unless you fetch at a higher frequency. That's mostly why I'd want this. It's not a huge deal obviously, but would be a nice improvement.

Couldn't something like this be put into place to at least make a best attempt at sorting the content by when it was published?

Sort By: If PublishedDate is determined to be well-structured/reliable, use PublishedDate else use FetchedDate

It doesn't have to be perfect, but I think it would help to distribute the feed content.

olinorwell · 2022-05-28T16:14:41Z

I have to agree with @0x199x - sorting by PublishedDate then FetchedDate if PublishedDate is empty/'unreliable' would be a massive improvement. Even if it was optional, I think many people would immediately turn it on.

Currently I have a situation where I add a new personal blog for somebody, then at the top of my feed are effectively all the stuff they have ever written for the last few years, all listed as today, while I can clearly see the correct published date next to the title.

I think the issue here is that it depends on what type of RSS feeds you are subscribing to. If you are subscribed to hundreds and they are very frequently updated, then this issue wouldn't be so obvious due to the sheer number of items.

I subscribe to say 10 programmer blogs, who post every couple of months. All posts have a correct published date. If I add a new blog, I want the new blog's articles to sit in my 'timeline' on the correct dates they were published. Instead I get 'How to do XYZ, published May 2021' under 28 May 2022, 'How to do ABC, published Feb 2021', also under 28 May 2022, and so on, for typically 20+ items.

This seems like an obvious fix - when importing a post, take at face value the published date, and store that as the main date in the inbox. If it is empty, by all means use today.

sjthespian · 2022-09-08T21:37:34Z

For another use case for sorting by publication date, take a look at the feed at https://www.elephant.town/comic/rss. The author releases a new chapter every week of 25 pages or so. Since the pubDate is ignored, I'm ending up with a scrambled list of pages every week. For example, this is what I have after this week's update:

I basically end up ignoring FreshRSS, except as a notification of a new chapter, and have to read it on her web site.

Alkarex · 2022-09-09T08:51:08Z

For another use case for sorting by publication date, take a look at the feed at https://www.elephant.town/comic/rss. The author releases a new chapter every week of 25 pages or so. Since the pubDate is ignored, I'm ending up with a scrambled list of pages every week.

Interesting example @sjthespian . Actually, it should work just fine because FreshRSS does consider the publication date behinds the scenes when retrieving a batch of new articles. I have just tried and it worked fine for me.

I am not sure of where the problem comes from in your case, but I will try to keep an eye on it. It could be that the publications dates in the RSS feed were first wrong when you got them, and then later manually fixed. Or it might be that some chapters were published in a random order before some others.

With exactly one second of difference between the dates Sun, 28 Aug 2022 22:00:24 -0400, Sun, 28 Aug 2022 22:00:23 -0400, Sun, 28 Aug 2022 22:00:22 -0400 it looks like there is a manual process somewhere.

Anyway, I plan to add more sorting criteria in the near future, including alpha-numeric.

Frenzie · 2022-09-09T10:14:37Z

@Alkarex Actually 1.19.x and edge don't behave the same here. @sjthespian's screenshot is from 1.19.x or older, yours is from upcoming 1.20.

Edit: put another way, whatever that issue is, it's already fixed. ;-)

limdingwen · 2022-12-28T09:37:47Z

EDIT: This does not work. See below for a better patch.

Here is a quick hack to enable published-date sorting in the web UI. Change the line in app/Models/Entry.php from dateAdded to date.

Or if you're up for some git apply:

From de80b56e3a5eff74c4fd49fe9dd917085a883452 Mon Sep 17 00:00:00 2001
From: Lim Ding Wen <limdingwen@gmail.com>
Date: Wed, 28 Dec 2022 17:33:49 +0800
Subject: [PATCH] HACK: Sort by published date

I have no idea why this works.
---
 app/Models/Entry.php | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/Models/Entry.php b/app/Models/Entry.php
index 47fcf3b4..14a24c5b 100644
--- a/app/Models/Entry.php
+++ b/app/Models/Entry.php
@@ -507,7 +507,7 @@ class FreshRSS_Entry extends Minz_Model {
 	}
 
 	public function isDay(int $day, int $today): bool {
-		$date = $this->dateAdded(true);
+		$date = $this->date(true);
 		switch ($day) {
 		case FreshRSS_Days::TODAY:
 			$tomorrow = $today + 86400;
-- 
2.30.2

0001-HACK-Sort-by-published-date.patch

limdingwen · 2022-12-30T08:06:51Z

@Alkarex Actually 1.19.x and edge don't behave the same here. @sjthespian's screenshot is from 1.19.x or older, yours is from upcoming 1.20.

Edit: put another way, whatever that issue is, it's already fixed. ;-)

From my testing it seems that Normal view sorts by Fetch Date first, followed by Publish Date if the Fetch Date is the same.

limdingwen · 2022-12-30T09:55:57Z

Here's a better patch, however, no options menu for this yet because I don't need an options menu right now. This means this is not PR-ready, but hopefully this is useful to whoever wants this change.

This patch permanently changes the sorting method to published date. Tested with the web app and SQLite. Might not work with mobile apps.

Make a backup of FreshRSS.
In your copy of FreshRSS, type git apply and press enter.
Copy and paste below.
Press Ctrl-D.

From 03136e7e8890040c5539bfae55134b4205606104 Mon Sep 17 00:00:00 2001
From: Lim Ding Wen <limdingwen@gmail.com>
Date: Fri, 30 Dec 2022 17:50:15 +0800
Subject: [PATCH] Sort by published date instead of fetch date

Does not add an option for this.
---
 app/Controllers/indexController.php | 2 +-
 app/Models/Entry.php                | 2 +-
 app/Models/EntryDAO.php             | 6 +++---
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/app/Controllers/indexController.php b/app/Controllers/indexController.php
index 7fced48a..918235ec 100755
--- a/app/Controllers/indexController.php
+++ b/app/Controllers/indexController.php
@@ -87,7 +87,7 @@ class FreshRSS_index_Controller extends FreshRSS_ActionController {
                        if ($nbEntries >= FreshRSS_Context::$number) {
                                //We have enough entries: we discard the last one to use it for the next articles' page
                                ob_clean();
-                               FreshRSS_Context::$next_id = $lastEntry->id();
+                               FreshRSS_Context::$next_id = $lastEntry->date(true);
                        }
                        ob_end_flush();
                };
diff --git a/app/Models/Entry.php b/app/Models/Entry.php
index 47fcf3b4..14a24c5b 100644
--- a/app/Models/Entry.php
+++ b/app/Models/Entry.php
@@ -507,7 +507,7 @@ class FreshRSS_Entry extends Minz_Model {
        }
 
        public function isDay(int $day, int $today): bool {
-               $date = $this->dateAdded(true);
+               $date = $this->date(true);
                switch ($day) {
                case FreshRSS_Days::TODAY:
                        $tomorrow = $today + 86400;
diff --git a/app/Models/EntryDAO.php b/app/Models/EntryDAO.php
index cda51e5b..59d4065b 100644
--- a/app/Models/EntryDAO.php
+++ b/app/Models/EntryDAO.php
@@ -1050,7 +1050,7 @@ SQL;
                                throw new FreshRSS_EntriesGetter_Exception('Bad order in Entry->listByType: [' . $order . ']!');
                }
                if ($firstId !== '') {
-                       $search .= 'AND ' . $alias . 'id ' . ($order === 'DESC' ? '<=' : '>=') . ' ? ';
+                       $search .= 'AND ' . $alias . 'date ' . ($order === 'DESC' ? '<=' : '>=') . ' ? ';
                        $values[] = $firstId;
                }
                if ($date_min > 0) {
@@ -1123,7 +1123,7 @@ SQL;
                        . ($type === 't' || $type === 'T' ? 'INNER JOIN `_entrytag` et ON et.id_entry = e.id ' : '')
                        . 'WHERE ' . $where
                        . $search
-                       . 'ORDER BY e.id ' . $order
+                       . 'ORDER BY e.date ' . $order
                        . ($limit > 0 ? ' LIMIT ' . intval($limit) : ''));      //TODO: See http://explainextended.com/2009/10/23/mysql-order-by-limit-performance-late-row-lookups/
        }
 
@@ -1138,7 +1138,7 @@ SQL;
                        . 'INNER JOIN ('
                        . $sql
                        . ') e2 ON e2.id=e0.id '
-                       . 'ORDER BY e0.id ' . $order;
+                       . 'ORDER BY e0.date ' . $order;
 
                $stm = $this->pdo->prepare($sql);
                if ($stm && $stm->execute($values)) {
-- 
5.30.2

If the copy-paste doesn't work, try git apply <file name> from this instead:

0001-Sort-by-published-date-instead-of-fetch-date.patch

redactedscribe · 2023-02-06T00:42:13Z

Just installed FreshRSS. First thing I wanted was my imported feeds to be sorted by published date as a whole (not by published date and grouped based upon the time the feed was added). As a first impression, it makes the program seem like it doesn't order things correctly (even though I assume it does once FreshRSS starts to pull the feeds in daily).

aofilhox · 2023-05-01T13:30:07Z

Hello everybody!
I made things easy for us:

backup your FreshRSS installation;
download the compressed file at
https://aofilho.com.br/dev/apps/freshrss1210-dateorder.zip ;
unzip and copy to the root folder of the FreshRSS installation;
ready, streaming organized by publication date;
Enjoy!

Hugs to all!

Antonio Filho
A guy who simplifies problems.

Thanks to @limdingwen - I used your code/solution. :)

limdingwen · 2023-05-01T17:55:52Z

Thanks for simplifying things, @antonioof! 😉 I’m glad to have helped.

ThallesLazaro · 2024-05-02T15:48:50Z

Here's a better patch, however, no options menu for this yet because I don't need an options menu right now. This means this is not PR-ready, but hopefully this is useful to whoever wants this change.

This patch permanently changes the sorting method to published date. Tested with the web app and SQLite. Might not work with mobile apps.

Make a backup of FreshRSS.
In your copy of FreshRSS, type git apply and press enter.
Copy and paste below.
Press Ctrl-D.

From 03136e7e8890040c5539bfae55134b4205606104 Mon Sep 17 00:00:00 2001
From: Lim Ding Wen <limdingwen@gmail.com>
Date: Fri, 30 Dec 2022 17:50:15 +0800
Subject: [PATCH] Sort by published date instead of fetch date

Does not add an option for this.
---
 app/Controllers/indexController.php | 2 +-
 app/Models/Entry.php                | 2 +-
 app/Models/EntryDAO.php             | 6 +++---
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/app/Controllers/indexController.php b/app/Controllers/indexController.php
index 7fced48a..918235ec 100755
--- a/app/Controllers/indexController.php
+++ b/app/Controllers/indexController.php
@@ -87,7 +87,7 @@ class FreshRSS_index_Controller extends FreshRSS_ActionController {
                        if ($nbEntries >= FreshRSS_Context::$number) {
                                //We have enough entries: we discard the last one to use it for the next articles' page
                                ob_clean();
-                               FreshRSS_Context::$next_id = $lastEntry->id();
+                               FreshRSS_Context::$next_id = $lastEntry->date(true);
                        }
                        ob_end_flush();
                };
diff --git a/app/Models/Entry.php b/app/Models/Entry.php
index 47fcf3b4..14a24c5b 100644
--- a/app/Models/Entry.php
+++ b/app/Models/Entry.php
@@ -507,7 +507,7 @@ class FreshRSS_Entry extends Minz_Model {
        }
 
        public function isDay(int $day, int $today): bool {
-               $date = $this->dateAdded(true);
+               $date = $this->date(true);
                switch ($day) {
                case FreshRSS_Days::TODAY:
                        $tomorrow = $today + 86400;
diff --git a/app/Models/EntryDAO.php b/app/Models/EntryDAO.php
index cda51e5b..59d4065b 100644
--- a/app/Models/EntryDAO.php
+++ b/app/Models/EntryDAO.php
@@ -1050,7 +1050,7 @@ SQL;
                                throw new FreshRSS_EntriesGetter_Exception('Bad order in Entry->listByType: [' . $order . ']!');
                }
                if ($firstId !== '') {
-                       $search .= 'AND ' . $alias . 'id ' . ($order === 'DESC' ? '<=' : '>=') . ' ? ';
+                       $search .= 'AND ' . $alias . 'date ' . ($order === 'DESC' ? '<=' : '>=') . ' ? ';
                        $values[] = $firstId;
                }
                if ($date_min > 0) {
@@ -1123,7 +1123,7 @@ SQL;
                        . ($type === 't' || $type === 'T' ? 'INNER JOIN `_entrytag` et ON et.id_entry = e.id ' : '')
                        . 'WHERE ' . $where
                        . $search
-                       . 'ORDER BY e.id ' . $order
+                       . 'ORDER BY e.date ' . $order
                        . ($limit > 0 ? ' LIMIT ' . intval($limit) : ''));      //TODO: See http://explainextended.com/2009/10/23/mysql-order-by-limit-performance-late-row-lookups/
        }
 
@@ -1138,7 +1138,7 @@ SQL;
                        . 'INNER JOIN ('
                        . $sql
                        . ') e2 ON e2.id=e0.id '
-                       . 'ORDER BY e0.id ' . $order;
+                       . 'ORDER BY e0.date ' . $order;
 
                $stm = $this->pdo->prepare($sql);
                if ($stm && $stm->execute($values)) {
-- 
5.30.2

If the copy-paste doesn't work, try git apply <file name> from this instead:

0001-Sort-by-published-date-instead-of-fetch-date.patch

Hello everybody! I made things easy for us:

backup your FreshRSS installation;

download the compressed file at
https://aofilho.com.br/dev/apps/freshrss1210-dateorder.zip ;

unzip and copy to the root folder of the FreshRSS installation;

ready, streaming organized by publication date;

Enjoy!

Hugs to all!

Antonio Filho A guy who simplifies problems.

Thanks to @limdingwen - I used your code/solution. :)

A fix in PHP 8

Anyone who has a problem with the error: PHP Fatal error: Uncaught TypeError: Cannot assign int to property FreshRSS_Context::$next_id of type string in /home3/gravad70/*****/app/Controllers/indexController.php:97

In the part you change it to: FreshRSS_Context::$next_id = $lastEntry->date(true);

Put strval() in $lastEntry->date(true) and it will work perfectly, see below how it will look:

FreshRSS_Context::$next_id = strval($lastEntry->date(true));

It has :)

Alkarex added the Documentation 📚 label Oct 27, 2018

pattems mentioned this issue Nov 13, 2018

Episodes from a newly added feed show up under the today category #2119

Closed

pattems mentioned this issue Dec 1, 2018

Update Readme.conf #2158

Merged

aledeg mentioned this issue Oct 2, 2020

Can I sort my feed by date of publication by default? #3204

Closed

Alkarex mentioned this issue Aug 31, 2022

[BUG] Items not keeping sort by date after a new subscription is added #4405

Closed

math-GH added FR: sort articles Sort feeds by published Date Feature Request ideas for new features labels Sep 3, 2022

Alkarex mentioned this issue Nov 13, 2023

Problem with filtering by date #5864

Closed

flashlab mentioned this issue May 27, 2024

How do I tell FreshRSS to sort articles by publication date, not date added? #2596

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need help for changing the feed sort order from lastSeen to the publish date #2083

Need help for changing the feed sort order from lastSeen to the publish date #2083

hanan-cohen commented Oct 27, 2018

Alkarex commented Oct 27, 2018

Alkarex commented Oct 27, 2018 •

edited

hanan-cohen commented Oct 27, 2018

Alkarex commented Oct 27, 2018 •

edited

tetsuhou commented Dec 2, 2020

ghost commented Apr 21, 2022

olinorwell commented May 28, 2022 •

edited

sjthespian commented Sep 8, 2022

Alkarex commented Sep 9, 2022 •

edited

Frenzie commented Sep 9, 2022 •

edited

limdingwen commented Dec 28, 2022 •

edited

limdingwen commented Dec 30, 2022

limdingwen commented Dec 30, 2022 •

edited

redactedscribe commented Feb 6, 2023

aofilhox commented May 1, 2023

limdingwen commented May 1, 2023

ThallesLazaro commented May 2, 2024

Need help for changing the feed sort order from lastSeen to the publish date #2083

Need help for changing the feed sort order from lastSeen to the publish date #2083

Comments

hanan-cohen commented Oct 27, 2018

Alkarex commented Oct 27, 2018

Alkarex commented Oct 27, 2018 • edited

hanan-cohen commented Oct 27, 2018

Alkarex commented Oct 27, 2018 • edited

tetsuhou commented Dec 2, 2020

ghost commented Apr 21, 2022

olinorwell commented May 28, 2022 • edited

sjthespian commented Sep 8, 2022

Alkarex commented Sep 9, 2022 • edited

Frenzie commented Sep 9, 2022 • edited

limdingwen commented Dec 28, 2022 • edited

limdingwen commented Dec 30, 2022

limdingwen commented Dec 30, 2022 • edited

redactedscribe commented Feb 6, 2023

aofilhox commented May 1, 2023

limdingwen commented May 1, 2023

ThallesLazaro commented May 2, 2024

A fix in PHP 8

Alkarex commented Oct 27, 2018 •

edited

Alkarex commented Oct 27, 2018 •

edited

olinorwell commented May 28, 2022 •

edited

Alkarex commented Sep 9, 2022 •

edited

Frenzie commented Sep 9, 2022 •

edited

limdingwen commented Dec 28, 2022 •

edited

limdingwen commented Dec 30, 2022 •

edited