New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefix keys and tags #46

Closed
ArtemGoutsoul opened this Issue Aug 28, 2013 · 13 comments

Comments

Projects
None yet
5 participants
@ArtemGoutsoul

ArtemGoutsoul commented Aug 28, 2013

Seems like a great class, but sometimes the same server setup is used for multiple applications, which means that in it's current form the class is not usable. It would be great to have support for a key/tag prefix.

I. e. so that these prefixes:

const PREFIX_KEY      = 'zc:k:';
const PREFIX_TAG_IDS  = 'zc:ti:';

would actually be prefixed some an optional value fetched from configuration, e.g. named "prefix".

@rgoytacaz

This comment has been minimized.

rgoytacaz commented Aug 28, 2013

isnt it available in as inside the cache tag?

@ArtemGoutsoul

This comment has been minimized.

ArtemGoutsoul commented Aug 28, 2013

If I understood you correctly, you mean adding an extra tag? This would mean that this class would not be able to replace existing cache backends that have this functionality. One would need to go through all the code and attach a prefix or extra tag to all cache handling calls.
Most other cache backends have a possibility to add a prefix just once, and this way for example when you call clean() - it will only clean up the keys with this prefix.

@colinmollenhour

This comment has been minimized.

Owner

colinmollenhour commented Aug 28, 2013

There are much better ways to support multiple applications and I think running multiple applications on the same database on the same server is just a bad idea.. The best is to use multiple instances of Redis (better separation of memory allocation, less bottleneck on one CPU core, independent configurations, etc..). The second best is to use separate Redis databases (e.g. 0,1,2,...). If for some reason you don't like those options then I suggest using a prefix in your application. Magento does just this using a thing called "abstraction". ;)

@ArtemGoutsoul

This comment has been minimized.

ArtemGoutsoul commented Aug 28, 2013

It's not really a question of best practices (which we could discuss to no end ;) ). It's more of a question of existing environments and existing cache backends that do support key/tag prefix. At the moment one cannot simply replace another zend cache backend (e.g. another redis or file based one) with the cm cache backend.

Separate redis databases is theoretically a good idea, but in practice one would have to establish some sort of mapping between the individual applications and the redis databases since redis databases have to be named as consecutive integers, which means that there's yet another thing to manage during deployment and maintenance.

And yes sure, one can implement yet another abstraction layer between Zend Cache and the application, but I thought Zend Cache - was that abstraction layer :).

@ArtemGoutsoul

This comment has been minimized.

ArtemGoutsoul commented Aug 28, 2013

The prefix could be implemented in such a way that without it - the code would behave exactly the same way it behaves now, at the same time if configuration would have the prefix specified, the class would support one more use case - a large amount (in my case about a 100) of small completely independent applications using the same redis server.

@rgoytacaz

This comment has been minimized.

rgoytacaz commented Aug 28, 2013

I think adding the prefix functionally also brings the Backend_Redis on
par with current file/memcache/apc implementations which support that.

Would it just be as simple as fetching it from the xml and prefix
every key operation with it?

@colinmollenhour

This comment has been minimized.

Owner

colinmollenhour commented Aug 29, 2013

I fail to see how it is that much easier to set different 'prefix' per cache instance but not a different 'database'.. Just pretend that 'database' is actually 'prefix' and that the prefix must be an int and there you go. :-)

If you are already using a primary key for users perhaps that would work if the number of users is below the magic 1024. If not, then searching for the first unused database on signup should be pretty trivial. If you need more than 1024 then use incrementing port number and database. This is just how I would do it unless you just use very few keys per user in which case I think I'd just use my own wrapper to add a prefix.

Using a prefix means that one user causing a "flushdb" would flush all users cache, but this is not the case with 'database'. Also you can easily see using 'info' command how many keys each user is using. Lastly, if the prefix is very long it is going to make your key names all longer which means more memory wasted.

It may be that most zend backends support a prefix, but it isn't at all consistent between backends so I wouldn't call it an official feature and it just seems wrong to implement this feature as part of the backend.. Doh, just looked and guess what, the frontend supports a "cache_id_prefix"..

@rgoytacaz

This comment has been minimized.

rgoytacaz commented Aug 29, 2013

Uh Oh, forgot about the flushing part of it. You are right, its not optimal.

@ArtemGoutsoul

This comment has been minimized.

ArtemGoutsoul commented Aug 29, 2013

@colinmollenhour thanks a bunch for pointing me to the frontend cache_id_prefix! this will solve some of the use cases. However, the "clean all" call will have to be handled differently anyway since cache_id_prefix will probably not affect it.

@rgoytacaz

This comment has been minimized.

rgoytacaz commented Aug 29, 2013

Maybe creating an additional tag that would be present in all entries that
would contain the prefix, that way you can change the flushing to
invalidate the prefix tag, effectively cleaning only that store.

@bery

This comment has been minimized.

bery commented Jan 20, 2016

Opening an old issue, but just came across something interesting that is actually missing in the configuration example.

You should be able to use the built-in Magento cache prefix so you can protect multiple environments and/or applications in the same database like this

<config>
    <global>
        <cache>
            <backend>Mage_Cache_Backend_Redis</backend>
            <backend_options>
                <server>10.241.4.42</server> <!-- or absolute path to unix socket -->
                ..........
            </backend_options>
            <prefix>local_</prefix>
        </cache>
....
</global>

this produces something like

zc:k:local_Zend_Locale

Note that the prefix is included in the key's name.

@colinmollenhour

This comment has been minimized.

Owner

colinmollenhour commented Jan 20, 2016

Read the discussion, the proper place to set the prefix is the frontend not the backend and the frontend already supports cache_id_prefix.

@elachino

This comment has been minimized.

elachino commented Jul 19, 2017

We had same problem using Lesti_FPC, and we resolved how say @bery, adding tag in fpc.xml file:

<prefix>local_</prefix>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment