SHDICT enhancement proposal #97

Open
Thomas12 opened this Issue Apr 2, 2012 · 2 comments

Projects

None yet

3 participants

@Thomas12
Thomas12 commented Apr 2, 2012

Hi,

the current shdict implementation is nice and fast but lacks some important features.

Here a proposal for a shm-dict storage enhancement which would bring openresty's shdict implementation directly into the top-range of shm-caching due to its versatility, APIs or other custom enhancements should not be needed anymore:

sd:set(123456, 'bla');
sd:setchild(123456, 'age', 15);
sd:setchild(123456, 'name', 'John');
sd:set_alias_key(123456, 'Employee Level 4');
sd:commit(); --to do all accumulated tasks in one blocked transaction (simply means all above happens while your shdict-mutex is continuously closed for others)

ret=sd:get(123456);
pkey=ret['pkey']; --123456
akey=ret['akey']; --Employee Level 4
val=ret['val']; --bla
age=ret['age']; --15
name=ret['name']; --John

also works by alias key:

ret2=sd:get('Employee Level 4');
pkey=ret2['pkey']; --123456
akey=ret2['akey']; --Employee Level 4
val=ret2['val']; --bla
age=ret2['age']; --15
name=ret2['name']; --John

The indices pkey, akey and val are "reserved".

sd:del(123456); --automatically deletes all childs, and the alias key "Employee Level 4"

or

sd:del('Employee Level 4'); --automatically deletes all childs, and main key 123456

param='>=';
ret=sd:get(123455, param);

param can be '<=', '>=', '=' (default '=') to find the exact key or the next bigger one or the next smaller one until the best matching key is finally found, if no matching key exists, nil is returned.

Easily iterate over all keys, even over key-series with gaps in it as the next-best is found:

next_key=0;
while ret=sd:get(next_key, '>=') do
if(ret==nil or next_key>100) then break; end;
ngx.say('Found key: '..ret['pkey']..'\n');
next_key=ret['pkey']+1;
end;

What do you think?

Thomas

Owner
agentzh commented Apr 4, 2012

These features are interesting! They remind me of those Redis commands ;)

We're going to open the C API of ngx_lua's shared_dict so that people
(or ourselves) can write custom Nginx C modules to extend it or even
replace the implementation with something completely new :)

I'd really hope to keep the default shared_dict implementation in the
ngx_lua core as simple as possible :) And adding the capability of
extending or overriding the default engine will not stop us from
adding new features freely :)

Thanks!
-agentzh

rik1083 commented Apr 21, 2012

This is a very usefule feature indeed

I would really like something to persist the DICT across restarts
e.g. dump contents to file periodically, and load from file on new start up

Or even simpler, just a way to list all the key:val without having to know al the key's externally, in order to script my own backups

foreach (entry e in DICT) {
print(e.Key .. ":" .. e.Value)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment