Skip to content

Commit 17b349e

Browse files
agentzhagentzh
committed
feature: added new methods safe_set and safe_get to ngx.shared.DICT objects, which never override existing unexpired items but immediately return nil and a "no memory" string message when running out of storage. thanks Matthieu Tourne for requesting this.
1 parent 835aea9 commit 17b349e

File tree

5 files changed

+200
-1
lines changed

5 files changed

+200
-1
lines changed

‎README

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3832,8 +3832,12 @@ Nginx API for Lua
38323832

38333833
* set
38343834

3835+
* safe_set
3836+
38353837
* add
38363838

3839+
* safe_add
3840+
38373841
* replace
38383842

38393843
* incr
@@ -3994,6 +3998,23 @@ Nginx API for Lua
39943998

39953999
See also ngx.shared.DICT.
39964000

4001+
ngx.shared.DICT.safe_set
4002+
syntax: *ok, err = ngx.shared.DICT:safe_set(key, value, exptime?,
4003+
flags?)*
4004+
4005+
context: *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*,
4006+
content_by_lua*, header_filter_by_lua*, body_filter_by_lua*,
4007+
log_by_lua**
4008+
4009+
Similar to the set method, but never overrides the (least recently used)
4010+
unexpired items in the store when running out of storage in the shared
4011+
memory zone. In this case, it will immediately return "nil" and the
4012+
string "no memory".
4013+
4014+
This feature was first introduced in the "v0.7.18" release.
4015+
4016+
See also ngx.shared.DICT.
4017+
39974018
ngx.shared.DICT.add
39984019
syntax: *success, err, forcible = ngx.shared.DICT:add(key, value,
39994020
exptime?, flags?)*
@@ -4013,6 +4034,23 @@ Nginx API for Lua
40134034

40144035
See also ngx.shared.DICT.
40154036

4037+
ngx.shared.DICT.safe_add
4038+
syntax: *ok, err = ngx.shared.DICT:safe_add(key, value, exptime?,
4039+
flags?)*
4040+
4041+
context: *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*,
4042+
content_by_lua*, header_filter_by_lua*, body_filter_by_lua*,
4043+
log_by_lua**
4044+
4045+
Similar to the add method, but never overrides the (least recently used)
4046+
unexpired items in the store when running out of storage in the shared
4047+
memory zone. In this case, it will immediately return "nil" and the
4048+
string "no memory".
4049+
4050+
This feature was first introduced in the "v0.7.18" release.
4051+
4052+
See also ngx.shared.DICT.
4053+
40164054
ngx.shared.DICT.replace
40174055
syntax: *success, err, forcible = ngx.shared.DICT:replace(key, value,
40184056
exptime?, flags?)*

‎README.markdown

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3553,7 +3553,9 @@ The resulting object `dict` has the following methods:
35533553

35543554
* [get](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.get)
35553555
* [set](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.set)
3556+
* [safe_set](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.safe_set)
35563557
* [add](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.add)
3558+
* [safe_add](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.safe_add)
35573559
* [replace](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.replace)
35583560
* [incr](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.incr)
35593561
* [delete](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.delete)
@@ -3680,6 +3682,18 @@ Please note that while internally the key-value pair is set atomically, the atom
36803682

36813683
See also [ngx.shared.DICT](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT).
36823684

3685+
ngx.shared.DICT.safe_set
3686+
------------------------
3687+
**syntax:** *ok, err = ngx.shared.DICT:safe_set(key, value, exptime?, flags?)*
3688+
3689+
**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua**
3690+
3691+
Similar to the [set](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.set) method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return `nil` and the string "no memory".
3692+
3693+
This feature was first introduced in the `v0.7.18` release.
3694+
3695+
See also [ngx.shared.DICT](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT).
3696+
36833697
ngx.shared.DICT.add
36843698
-------------------
36853699
**syntax:** *success, err, forcible = ngx.shared.DICT:add(key, value, exptime?, flags?)*
@@ -3694,6 +3708,18 @@ This feature was first introduced in the `v0.3.1rc22` release.
36943708

36953709
See also [ngx.shared.DICT](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT).
36963710

3711+
ngx.shared.DICT.safe_add
3712+
------------------------
3713+
**syntax:** *ok, err = ngx.shared.DICT:safe_add(key, value, exptime?, flags?)*
3714+
3715+
**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua**
3716+
3717+
Similar to the [add](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT.add) method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return `nil` and the string "no memory".
3718+
3719+
This feature was first introduced in the `v0.7.18` release.
3720+
3721+
See also [ngx.shared.DICT](http://wiki.nginx.org/HttpLuaModule#ngx.shared.DICT).
3722+
36973723
ngx.shared.DICT.replace
36983724
-----------------------
36993725
**syntax:** *success, err, forcible = ngx.shared.DICT:replace(key, value, exptime?, flags?)*

‎doc/HttpLuaModule.wiki

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3433,7 +3433,9 @@ The resulting object <code>dict</code> has the following methods:
34333433
34343434
* [[#ngx.shared.DICT.get|get]]
34353435
* [[#ngx.shared.DICT.set|set]]
3436+
* [[#ngx.shared.DICT.safe_set|safe_set]]
34363437
* [[#ngx.shared.DICT.add|add]]
3438+
* [[#ngx.shared.DICT.safe_add|safe_add]]
34373439
* [[#ngx.shared.DICT.replace|replace]]
34383440
* [[#ngx.shared.DICT.incr|incr]]
34393441
* [[#ngx.shared.DICT.delete|delete]]
@@ -3558,6 +3560,17 @@ Please note that while internally the key-value pair is set atomically, the atom
35583560
35593561
See also [[#ngx.shared.DICT|ngx.shared.DICT]].
35603562
3563+
== ngx.shared.DICT.safe_set ==
3564+
'''syntax:''' ''ok, err = ngx.shared.DICT:safe_set(key, value, exptime?, flags?)''
3565+
3566+
'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*''
3567+
3568+
Similar to the [[#ngx.shared.DICT.set|set]] method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return <code>nil</code> and the string "no memory".
3569+
3570+
This feature was first introduced in the <code>v0.7.18</code> release.
3571+
3572+
See also [[#ngx.shared.DICT|ngx.shared.DICT]].
3573+
35613574
== ngx.shared.DICT.add ==
35623575
'''syntax:''' ''success, err, forcible = ngx.shared.DICT:add(key, value, exptime?, flags?)''
35633576
@@ -3571,6 +3584,17 @@ This feature was first introduced in the <code>v0.3.1rc22</code> release.
35713584
35723585
See also [[#ngx.shared.DICT|ngx.shared.DICT]].
35733586
3587+
== ngx.shared.DICT.safe_add ==
3588+
'''syntax:''' ''ok, err = ngx.shared.DICT:safe_add(key, value, exptime?, flags?)''
3589+
3590+
'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*''
3591+
3592+
Similar to the [[#ngx.shared.DICT.add|add]] method, but never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return <code>nil</code> and the string "no memory".
3593+
3594+
This feature was first introduced in the <code>v0.7.18</code> release.
3595+
3596+
See also [[#ngx.shared.DICT|ngx.shared.DICT]].
3597+
35743598
== ngx.shared.DICT.replace ==
35753599
'''syntax:''' ''success, err, forcible = ngx.shared.DICT:replace(key, value, exptime?, flags?)''
35763600

‎src/ngx_http_lua_shdict.c

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616

1717

1818
static int ngx_http_lua_shdict_set(lua_State *L);
19+
static int ngx_http_lua_shdict_safe_set(lua_State *L);
1920
static int ngx_http_lua_shdict_get(lua_State *L);
2021
static int ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx,
2122
ngx_uint_t n);
@@ -24,6 +25,7 @@ static ngx_int_t ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone,
2425
ngx_http_lua_shdict_node_t **sdp);
2526
static int ngx_http_lua_shdict_set_helper(lua_State *L, int flags);
2627
static int ngx_http_lua_shdict_add(lua_State *L);
28+
static int ngx_http_lua_shdict_safe_add(lua_State *L);
2729
static int ngx_http_lua_shdict_replace(lua_State *L);
2830
static int ngx_http_lua_shdict_incr(lua_State *L);
2931
static int ngx_http_lua_shdict_delete(lua_State *L);
@@ -34,6 +36,7 @@ static int ngx_http_lua_shdict_get_keys(lua_State *L);
3436

3537
#define NGX_HTTP_LUA_SHDICT_ADD 0x0001
3638
#define NGX_HTTP_LUA_SHDICT_REPLACE 0x0002
39+
#define NGX_HTTP_LUA_SHDICT_SAFE_STORE 0x0004
3740

3841

3942
ngx_int_t
@@ -296,9 +299,15 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L)
296299
lua_pushcfunction(L, ngx_http_lua_shdict_set);
297300
lua_setfield(L, -2, "set");
298301

302+
lua_pushcfunction(L, ngx_http_lua_shdict_safe_set);
303+
lua_setfield(L, -2, "safe_set");
304+
299305
lua_pushcfunction(L, ngx_http_lua_shdict_add);
300306
lua_setfield(L, -2, "add");
301307

308+
lua_pushcfunction(L, ngx_http_lua_shdict_safe_add);
309+
lua_setfield(L, -2, "safe_add");
310+
302311
lua_pushcfunction(L, ngx_http_lua_shdict_replace);
303312
lua_setfield(L, -2, "replace");
304313

@@ -727,6 +736,14 @@ ngx_http_lua_shdict_add(lua_State *L)
727736
}
728737

729738

739+
static int
740+
ngx_http_lua_shdict_safe_add(lua_State *L)
741+
{
742+
return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_ADD
743+
|NGX_HTTP_LUA_SHDICT_SAFE_STORE);
744+
}
745+
746+
730747
static int
731748
ngx_http_lua_shdict_replace(lua_State *L)
732749
{
@@ -741,6 +758,13 @@ ngx_http_lua_shdict_set(lua_State *L)
741758
}
742759

743760

761+
static int
762+
ngx_http_lua_shdict_safe_set(lua_State *L)
763+
{
764+
return ngx_http_lua_shdict_set_helper(L, NGX_HTTP_LUA_SHDICT_SAFE_STORE);
765+
}
766+
767+
744768
static int
745769
ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
746770
{
@@ -979,6 +1003,14 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags)
9791003

9801004
if (node == NULL) {
9811005

1006+
if (flags & NGX_HTTP_LUA_SHDICT_SAFE_STORE) {
1007+
ngx_shmtx_unlock(&ctx->shpool->mutex);
1008+
1009+
lua_pushnil(L);
1010+
lua_pushliteral(L, "no memory");
1011+
return 2;
1012+
}
1013+
9821014
ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ctx->log, 0,
9831015
"lua shared dict set: overriding non-expired items "
9841016
"due to memory shortage for entry \"%V\"", &name);

‎t/043-shdict.t

Lines changed: 80 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ use Test::Nginx::Socket;
88

99
#repeat_each(2);
1010

11-
plan tests => repeat_each() * (blocks() * 2 + 7);
11+
plan tests => repeat_each() * (blocks() * 2 + 15);
1212

1313
#no_diff();
1414
no_long_string();
@@ -1373,3 +1373,82 @@ GET /t
13731373
--- response_body
13741374
2048
13751375

1376+
1377+
1378+
=== TEST 58: safe_set
1379+
--- http_config
1380+
lua_shared_dict dogs 100k;
1381+
--- config
1382+
location = /test {
1383+
content_by_lua '
1384+
local dogs = ngx.shared.dogs
1385+
local i = 0
1386+
while i < 1000 do
1387+
i = i + 1
1388+
local val = string.rep(" hello", 10) .. i
1389+
local res, err = dogs:safe_set("key_" .. i, val)
1390+
if not res then
1391+
ngx.say(res, " ", err)
1392+
break
1393+
end
1394+
end
1395+
ngx.say("abort at ", i)
1396+
ngx.say("cur value: ", dogs:get("key_" .. i))
1397+
if i > 1 then
1398+
ngx.say("1st value: ", dogs:get("key_1"))
1399+
end
1400+
if i > 2 then
1401+
ngx.say("2nd value: ", dogs:get("key_2"))
1402+
end
1403+
';
1404+
}
1405+
--- pipelined_requests eval
1406+
["GET /test", "GET /test"]
1407+
--- response_body eval
1408+
my $a = "nil no memory\nabort at (353|705)\ncur value: nil\n1st value: " . (" hello" x 10) . "1\n2nd value: " . (" hello" x 10) . "2\n";
1409+
[qr/$a/, qr/$a/]
1410+
--- no_error_log
1411+
[error]
1412+
1413+
1414+
1415+
=== TEST 59: safe_add
1416+
--- http_config
1417+
lua_shared_dict dogs 100k;
1418+
--- config
1419+
location = /test {
1420+
content_by_lua '
1421+
local dogs = ngx.shared.dogs
1422+
local i = 0
1423+
while i < 1000 do
1424+
i = i + 1
1425+
local val = string.rep(" hello", 10) .. i
1426+
local res, err = dogs:safe_add("key_" .. i, val)
1427+
if not res then
1428+
ngx.say(res, " ", err)
1429+
break
1430+
end
1431+
end
1432+
ngx.say("abort at ", i)
1433+
ngx.say("cur value: ", dogs:get("key_" .. i))
1434+
if i > 1 then
1435+
ngx.say("1st value: ", dogs:get("key_1"))
1436+
end
1437+
if i > 2 then
1438+
ngx.say("2nd value: ", dogs:get("key_2"))
1439+
end
1440+
';
1441+
}
1442+
--- pipelined_requests eval
1443+
["GET /test", "GET /test"]
1444+
--- response_body eval
1445+
my $a = "nil no memory\nabort at (353|705)\ncur value: nil\n1st value: " . (" hello" x 10) . "1\n2nd value: " . (" hello" x 10) . "2\n";
1446+
[qr/$a/,
1447+
q{false exists
1448+
abort at 1
1449+
cur value: hello hello hello hello hello hello hello hello hello hello1
1450+
}
1451+
]
1452+
--- no_error_log
1453+
[error]
1454+

0 commit comments

Comments
 (0)