Persistence

[siennathesane]

It would be great if bigcache had a feature where it could save it's state to disk so it could be restarted. I'm thinking there could be a storage interface which defined the API and then the initial implementation could be some base layout, like JSON or something.

type IStorage interface {
    Prepare() error // lock the entire cache.
    Save() (bool, error) // save the state to disk, return ok, error
    Load() (bool, error) // load from wherever, return ok, error
}

This has come up a few times and it might be an interesting interface to explore.

[neal-zhu]

How about changing the interface to

type IStorage interface {
    Prepare() error // lock the entire cache.
    Save(io.Writer) (bool, error) // save the state to disk, return ok, error
    Load(io.Reader) (bool, error) // load from wherever, return ok, error
}

[cristaloleg]

@neal-zhu yep, this is definitely better. I'm thinking do we need to path context too?

[neal-zhu]

Using io.Writer makes us free to save our data anywhere we like(maybe local disk, or a remote node or main memory). So I think path context is unnecessary?

[siennathesane]

I'm totally open to modifications on it, when I wrote this it was to open discussion, so I'm not married to the final decision so much as it was to surface this is an interface we'd want to implement.

I do agree that we don't need a path context and the Reader/Writer implementation is more than enough for the initial use cases. Partially because then it allows for disk, streaming, or some other method.

[neal-zhu]

locking entire cache during persistence might not be acceptable, can we use a child process just like redis?

type IStorage interface {
    Save(string) chan error // fork a child process to dump entire cache
    Load(string) (bool, error) // load from wherever, return ok, error
}

[siennathesane]

I think having both a full-lock and a no-lock option would be good, but I think a full-lock could also be achieved upstream, so I'm open to a first pass and seeing what the community says. :)