All subscription items reappear

[justdanpo]

Not often but repeatedly I get the whole item list of some subscription, which hasn't been updated for a while, reappeared. The last one is https://archive.org/services/collection-rss.php?collection=ringe-raja-records

[justdanpo]

Oh, another one =( https://archive.org/services/collection-rss.php?collection=dustedwaxkingdom

[Gonzih]

That might be due to change in output format, so checksum does not match anymore.

[justdanpo]

I have one more subscription from archive.org https://archive.org/services/collection-rss.php?collection=dance-corps which wasn't affected this time, so I believe it's not about output format

[Gonzih]

Yeah, sorry, dunno why then this happened.
This is not an actively maintained project, I switched to my go implementation long time ago (which has intentional TTL on items in cache, so this would happen in that project by design).
So cant help here much, PRs are always welcome though.

[justdanpo]

Just for the record: dance-corps reappeared today. I've added extra logging and now I'm 100% sure hashes haven't been changed.

[Gonzih]

I remember there was a cleanup logic at some point in this project. But looking at the current state of https://github.com/Gonzih/feeds2imap.clj/blob/3119214f5fe8feb933bc7ebb62c445de09c80494/src/feeds2imap/db.clj that handles cache in sqlite I dont see anything. So i assume i removed it.

Have you tried looking at sqlite cache database? Trying to query it see if there are duplicate entries in it?