Support requesting delta feeds (RFC3229+feed)

[da2x]

When sending a If-None-Match: $etag header, tag along a A-IM: feed
header. Supporting servers should only return content that has been added
or changed since they sent the etag. content. Should have no affect on
servers who doesn’t support this. Saves bandwidth when supported.

Supported by ExpressionEngine, Textpattern, and Liferea. I’m working on adding support to more feed readers and hope to add it to WordPress soon.

[mblaney]

Hi @Aeyoun that is an interesting header to add. I like the idea of it but I don't think this change is enough. If it does get used and a delta is returned, that would replace the current cache for the feed. To make it useful, when a delta is returned it would need to be appended to the start of the cache.

[JanPetterMG]

Working on a If-None-Match/ETag implementation myself atm, not using Simplepies cache, but my own custom solution, on top of the Simplepie parser. We might have similar issues/cases, therefore I'm asking:

• Wouldn't appending the change to the top of the cache, make the cache grow infinitely? How do we know what to delete, or what to keep, without having to do an clean request again?

I've seen feeds never containing more than 1 item at a time, and others with thousands. There is also those with a internal time limit, basically making the item count vary on a daily basis.

There may be usage cases I haven't think'd about, but I think there's two groups of developers in this world:

1. Parsing feeds regularly. You'll probably want the changes only. (the rest is wasted resources)
2. Spamming Simplepie way too frequently, requesting a single feed (eg. on demand, every time a blog is visited). You'll probably want the last 10 items or so.

How about implementing a method to disable A-IM? eg. $this->preserveCache(bool);

[da2x]

If it does get used and a delta is returned, that would replace the current cache for the feed.

I musth ave misunderstood what SimplePie does. I thought there already was a feed cache? and that only new items (e.g. a delta) would be needed. This PR may be worthless if this isn’t the case.

How about implementing a method to disable A-IM? eg. $this->preserveCache(bool);

No cache, no etag, no A-IM header. No need for a separate switch, is there?

[JanPetterMG]

Simplepie already has an caching-mechanism integrated, so this PR is highly relevant.
I'm not using it, for variety reasons, so I don't know much details, other than it's disk based...

[mblaney]

@Aeyoun the cache for a feed is replaced every time you make a successful request. All I'm saying is that adding this header could have a consequence that hasn't been considered, ie replacing the current cache with a shorter version. I don't have a problem with that, but it either needs to be optional or do the extra work of adding the delta to the existing cache.

@JanPetterMG yes the cache would keep growing if this was done, but disk space is cheap ;-). If someone wanted to make this change you could always drop entries off the other end. Personally I currently use the SimplePie cache as temporary storage and store every feed entry permanently in a database, and I'm surprised how little space it all takes. (that is only for maybe 100 feeds though...)

So after saying that, working on caching probably isn't worth anyone's time, but a simple flag to make this optional would be fine and some of us would use it. Maybe not preserveCache but something like acceptDeltas?

[ifsnop]

@JanPetterMG https://github.com/JanPetterMG yes the cache would keep
growing if this was done, but disk space is cheap ;-). If someone wanted to
make this change you could always drop entries off the other end.
Personally I currently use the SimplePie cache as temporary storage and
store every feed entry permanently in a database, and I'm surprised how
little space it all takes. (that is only for maybe 100 feeds though...)

A patch was submitted some time ago that deletes the oldest entries from
the MySQL cache :)

[da2x]

I’ll close this as I thought SimplePie already did the required caching.