Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Is it possible to select a non-default db? #194

Closed
ragulka opened this Issue Mar 28, 2013 · 12 comments

Comments

Projects
None yet
7 participants

ragulka commented Mar 28, 2013

I'd like to selec another redis database (1, not the default 0) in my test environment. Is it possible with Kue? I didn't find anything in the docs.

http://rediscookbook.org/multiple_databases.html
https://github.com/mranney/node_redis#usage

Contributor

drudge commented Mar 28, 2013

@ragulka It is not currently possible. We're investigating this as a possibility, but I believe the idea of databases in redis is going away the future.

@ghost

ghost commented Jul 17, 2013

Isn't this possible by using a custom createClient function as specified in the Readme?

  var kue = require('kue')
    , redis = require('redis');

  kue.redis.createClient = function() {
    var client = redis.createClient(1234, '192.168.1.2');
    client.select(1)  // your database here
    client.auth('password');
    return client;
  };

The above implementation does run the risk of running into synchronous vs async issues, which unfortunately is a bit of a problem in node-redis as well (see mranney/node_redis#460). While it's possible to rewrap all createClient functions internally to have callbacks, it would seem that the domain of this problem would be in node-redis.

For what it's worth...from the creator of Redis:

"I consider Redis multiple database errors my worst decision in Redis design at all...without any kind of real gain, it makes the internals a lot more complex." – Salvatore Sanfilippo
https://groups.google.com/forum/#!topic/redis-db/vS5wX8X4Cjg

@behrad behrad closed this Jan 27, 2014

v4l3r10 commented May 22, 2014

+1 for db select !

Collaborator

behrad commented May 22, 2014

You can find out that using multiple databases in one redis instance is discouraged in redis community. So we won't go to support a deprecated redis feature.
But if you want to have isolation of Queues and Jobs you can have a look at LearnBoost#274

johnobe commented Jun 30, 2014

+1.

@behrad I understand that using multiple databases is discouraged, but are they really deprecated?

3rd party redis hosts, such as RedisToGo, utilize multiple databases.

FWIW, I'm using RedisToGo with Heroku.

inca commented Jul 1, 2014

@behrad I didn't heard of such a thing. Furthermore, multiple databases are actually encouraged in some cases (easy FLUSHDB for volatile data, super-easy sharding in contrast to key-namespaces, etc).

Collaborator

behrad commented Jul 1, 2014

I'd like to ask what makes you use them instead of key space isolation !? In kue you can config different queue prefixes on top a single redis instance to provide multi-tenant queues, doesn't it help !?

@johnobe @inca Please read this topic (https://groups.google.com/forum/#!topic/redis-db/u1IgtiWN0Ww) where Josiah (one of redis contributors) says:

  1. It adds needless complexity.
  2. It will go away in the future.
  3. Everything that you want to do with databases can be done via other methods

you may also want to read this: http://comments.gmane.org/gmane.comp.db.redis.general/12532

Collaborator

behrad commented Jul 1, 2014

You can easily do this:

var kue = require('kue');
var jobs = kue.createQueue({ prefix: 'default-queue' });
// jobs.process|create....

AND in another process

var kue = require('kue');
var jobs = kue.createQueue({ prefix: 'other-business-queues' });
// jobs.process|create....

AND in some other process

var kue = require('kue');
var jobs = kue.createQueue({ prefix: 'my-tests' });
// jobs.process|create....

The only limitation currently is that different queue objects cannot exist in a single node process since they are singletons.

johnobe commented Jul 1, 2014

@behrad With 3rd party services like RedisToGo, I don't have any control over how the redis service is provisioned or managed. When I request their service to be provisioned they give me a host, port, database name and password that I use within my application.

RedisToGo most likely has multiple customers sharing the same redis process (especially on smaller plans for apps that may not consume as much data), but the data for each customer is hidden from other customers through the use of multiple databases.

It would be nice (from my standpoint) if RedisToGo provisioned a dedicated redis process for each customer database, so processes are the data boundaries, not the databases themselves (this was suggested by Pieter in the thread you cited), but from perspective of RedisToGo, using multiple databases saves memory and cuts down on process overhead.

To be clear, setting different prefixes like you suggested doesn't help in this scenario because the correct database must be selected before an operation that would use the prefix is performed.

Collaborator

behrad commented Jul 1, 2014

and what if you could say

var kue  = require('kue');
var jobs = kue.createQueue({
  redis: {
    port: 1234,
    host: '10.0.50.20',
    db: 2, // will trigger redis.select( 2, ...) to use the second db in your redis instance
    auth: 'password',
});

johnobe commented Jul 1, 2014

That would work. That's very similar to how connect-redis is configured.

Collaborator

behrad commented Jul 1, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment