Use ssb-blobs-purge, to allow storing all blobs clientside

[Powersource]

https://github.com/staltz/ssb-blobs-purge

Would resolve Powersource/ssb-webextension-demo#4

@arj03 do you think this would be a good idea?

[arj03]

Yes that would be a great idea! @Powersource

[KyleMaas]

How would you figure out what to use for a storage limit?

[arj03]

I guess a sane default like maybe 100mb and a way to set this would be a good start I think.

[KyleMaas]

It looks like this has to be done at the ssb-browser-core level. What is the difference between simple-blobs.js and full ssb-blobs? Because ssb-blobs-purge requires ssb-blobs (or something like it), and simple-blobs.js does not seem to have the APIs it needs to work.

[arj03]

Yeah that might be a problem. simple-blobs is a complete replacement of the blobs part of the SSB stack because the existing was littered with fs stuff. So i guess this tasks is a lot more difficuelt because of this sadly :(

[KyleMaas]

Hm. Well, maybe the solution, then, is to try to duplicate (in simple-blobs) the APIs from ssb-blobs which ssb-blobs-purge needs?

[arj03]

Yes that would a good way to solve this :-)

[KyleMaas]

After attempting this, I'm not sure this is possible. I got close, but there are two problems I'm running into which I don't see being able to overcome:

• random-access-file doesn't seem to provide a way to see if a file already exists or get the size of a file without creating it. ssb-blobs-purge uses the size as part of its heuristic.
• As far as I can tell, the timestamp of the blob is not stored. This is also used by the heuristic.

I've pushed my changes to my ssb-blobs-purge branch, but I think that's about as far as I'm likely to be able to get. Sorry!

[arj03]

Don't be sorry! The missing features in random-access-file might be added later. It's a really good starting point.

For completeness this is the branch.

[KyleMaas]

So, according to:

random-access-storage/random-access-file#25

...we could potentially check if the blob is zero bytes and use that. We would still need to figure out the timestamp of the blob. Is there a good (quick, efficient) way to do that?

[KyleMaas]

Getting closer. I think I've got a workaround for simple-blobs. However, now I'm blocked on this:

ssbc/ssb-backlinks#22

[arj03]

The purge module needs to be converted to db2. Meaning backlinks should be replaced with a seek query. Probably a prefix query (like key).

[KyleMaas]

ssbc/ssb-blobs-purge#1

[KyleMaas]

That's live. Ran into another problem:

ssbc/ssb-blobs-purge#3