You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Moreover, when you do EVE(cache=None) you may expect that pycrest do not cache anything instead of using a default "DictCache" that you may not want and that will make your process size in RAM grow a lot...
Why :
Because in some use, you don't want to cache data for multiple reasons.
Simple case: getting all EVE Online orders, in all regions (through region.marketOrderAll). if you take just the 64 k-space regions, you will save in the process memory / memcached / other, all the data.
In the case you do it in a batch, you don't really care if it's cached or not, but you care about CPU perf and RAM, and the latter is the issue here.
If you loop through every regions, with a DictCache, you'll end with a python process size from 1.5Go to 2Go, with 90% of this size used by the cache (in my case, the process grew to 2.4Go with it, but only ~250Mo with the above lines commented)
I could have used a specific cache system (memcached, redis, or anything else), but i will just move ~1.5-2Go (useless) data from my python process to another process, so it's not the right solution.
Possible Solution(s) :
if cache=None then set no cache / DummyCache instead of using a "hidden default DictCache"
or change the get method to something like def get(self, resource, params={}, caching=True):, to be able to disable manually the cache on some requests
The text was updated successfully, but these errors were encountered:
Kyria
changed the title
Allowing a "no cache" option, and no default to "dict cache"
Allowing a "no cache" option instead of default DictCache
Sep 8, 2016
I totally can find myself in the suggested solutions (None should indeed imply absence of any cache and DictCache should be default). Being able to disabled cache per request (taking CREST it's wierd caching timers into account) would certainly add value as well.
At the moment, if we want to use pycrest without caching, we have 2 options :
Moreover, when you do
EVE(cache=None)
you may expect that pycrest do not cache anything instead of using a default "DictCache" that you may not want and that will make your process size in RAM grow a lot...Why :
Because in some use, you don't want to cache data for multiple reasons.
Simple case: getting all EVE Online orders, in all regions (through region.marketOrderAll). if you take just the 64 k-space regions, you will save in the process memory / memcached / other, all the data.
In the case you do it in a batch, you don't really care if it's cached or not, but you care about CPU perf and RAM, and the latter is the issue here.
If you loop through every regions, with a DictCache, you'll end with a python process size from 1.5Go to 2Go, with 90% of this size used by the cache (in my case, the process grew to 2.4Go with it, but only ~250Mo with the above lines commented)
I could have used a specific cache system (memcached, redis, or anything else), but i will just move ~1.5-2Go (useless) data from my python process to another process, so it's not the right solution.
Possible Solution(s) :
cache=None
then set no cache / DummyCache instead of using a "hidden default DictCache"def get(self, resource, params={}, caching=True):
, to be able to disable manually the cache on some requestsThe text was updated successfully, but these errors were encountered: