Add stored_data and store_data methods

[fractaledmind]

I think I want to add stored_data and store_data to Workflow class. These methods would exactly mirror cached_data and cache_data, but the data would be saved to the workflow's storage dir and no max_age would be required (i.e. max_age=0 is the permanent default).

However, I was wondering if there was some larger reason that you didn't originally put these methods in that would keep you from accepting a later pull-request?

Also, I would prefer to serialize the data into JSON, as mentioned in Issue 16.

[deanishe]

As mentioned in #16, JSON doesn't support common Python objects. That's the reason Workflow doesn't use it except to store the settings (the settings file has to be in one human-readable format or another, and JSON allows a nicer API than, e.g., ConfigParser).

The lack of stored_data() and store_data() methods is mostly due my belief that cache_data(name, data) and cached_data(name, max_age=0) achieve the same results (albeit using a different directory).

They should probably be added as a way to put data in the data directory, but I think they would also have to use pickle, at least by default: the choice to use a different on-disk format needs to be consciously made by the user because it probably won't accept arbitrary data.

Both the store_data and cache_data methods could be modified to accept serializer and extension arguments to use any custom serialiser that supports the dump(obj, fp)/load(fp) interface.

[fractaledmind]

I've added a Pull Request with these features added. I didn't add an extension arg tho; I simply use the serializer name as the extension. pickle is the default serializer for all 4 data storage methods, but user's can set there own serializer on instantiation. That seems a good balance of safe defaults, but extensibility.

[fractaledmind]

PS. This is all coming up because I'm rewriting ZotQuery from the ground up, and I'm really trying to take advantage of Alfred Workflow this time. I'm also trying to have cleaner, smarter data storage and models. The logic of the cached_data method is genius, since you can grab or create data with one call, but I want to store ZotQuery data in permanent storage and in JSON format. Thus, my desire for the changes. However, I reiterate, this is a great tool, so thank you.

[deanishe]

WRT permanent storage: data in the cache dir only gets deleted when a user explicitly deletes their caches using some tool like OnyX.

It's generally a dumb idea to delete the cache unless something is going wrong: the data is there to speed things up.

The difference is mostly semantic: data in caches is ephemeral or can be easily recreated; data stored in data directories is user-generated and can't be automatically recreated (e.g. settings).

I'm certainly open to using the data directory as the storage for the cache_ functions if the potential deletion of the data is an issue. Workflow does a fair job of managing the data (it isn't going to fill the cache with endless gigabytes of shite unless actively abused).

[deanishe]

BTW, what did you decide to do regarding search? Are you going to try sqlite?

[deanishe]

Added store_data() and stored_data() methods, plus SerializerManager

See v1.8.