Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto-generate the command table from JSON files #9656

Merged
merged 63 commits into from Dec 15, 2021
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
11193cd
WIP gen command table
guybe7 Sep 15, 2021
bd3730b
refactor
guybe7 Oct 27, 2021
38a3805
cont
guybe7 Oct 29, 2021
f7d288d
minimise preprocessor usage
guybe7 Oct 29, 2021
827719e
restore help.h (for now)
guybe7 Oct 29, 2021
d3ab327
cont
guybe7 Nov 3, 2021
31f13dd
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Nov 3, 2021
6739408
cont
guybe7 Nov 3, 2021
49c29bc
cont 2
guybe7 Nov 10, 2021
f25f3e8
typo
guybe7 Nov 10, 2021
8e7c5ef
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Nov 10, 2021
c49c859
some fixes
guybe7 Nov 10, 2021
da1605b
cont 3
guybe7 Nov 11, 2021
1111ba3
cont 4
guybe7 Nov 17, 2021
0df1448
cont 5
guybe7 Nov 17, 2021
873aa25
fix module tests
guybe7 Nov 17, 2021
6173023
fix eval
guybe7 Nov 17, 2021
60e8ad1
cont
guybe7 Nov 17, 2021
71c5af6
cont 2
guybe7 Nov 22, 2021
038d858
cont 3
guybe7 Nov 22, 2021
3206d33
almost done
guybe7 Nov 24, 2021
1b918a7
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Nov 24, 2021
16a8595
almost done 2
guybe7 Nov 24, 2021
3e999cb
remove return_type for the moment
guybe7 Nov 24, 2021
a693243
typo
guybe7 Nov 24, 2021
bb267a5
mostly refactoring in module, add history API
guybe7 Nov 24, 2021
3ba6d81
more APIs
guybe7 Nov 24, 2021
436ef4b
add args API
guybe7 Nov 28, 2021
fc4002d
some more fixes
guybe7 Nov 28, 2021
b2d81c8
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Nov 28, 2021
b493b7a
remove lcsGetKeys
guybe7 Nov 28, 2021
dfc5383
fix memleak
guybe7 Nov 28, 2021
1ab3f34
CR fixes 1
guybe7 Nov 30, 2021
d12c769
fix ACL test
guybe7 Nov 30, 2021
3f924b8
CR fixes 2
guybe7 Dec 1, 2021
eb31e91
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Dec 1, 2021
d317b53
fix CONFIG SET arity
guybe7 Dec 1, 2021
2e467b8
re-gen commands.c
guybe7 Dec 1, 2021
a235623
fix addACLLogEntry
guybe7 Dec 1, 2021
3d268a3
literals uppercase, placeholders lowercase
guybe7 Dec 1, 2021
0e6832d
reference key arg to key-spec
guybe7 Dec 1, 2021
8341d27
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Dec 3, 2021
b436bc9
CR fixes 2
guybe7 Dec 3, 2021
707931e
update jsons with new Redis functions
guybe7 Dec 3, 2021
59fa55b
fix test
guybe7 Dec 3, 2021
87a4f7f
COMMAND LIST args + cleanup of non-optional pure-tokens
guybe7 Dec 6, 2021
fd58bc6
move function command declarations to server.h
guybe7 Dec 6, 2021
87f3218
apply module API doc standards
guybe7 Dec 6, 2021
4070176
more fixes
guybe7 Dec 8, 2021
49b148f
cont
guybe7 Dec 8, 2021
dfcf81b
Merge remote-tracking branch 'origin/unstable' into gen_command_code
guybe7 Dec 8, 2021
31ec2f3
remove unused arg from moduleAPI
guybe7 Dec 15, 2021
58bc08e
add CMD_MODULE to COMMAND INFO
guybe7 Dec 15, 2021
be1989e
Yossi's comments
guybe7 Dec 15, 2021
6264460
add args where needed
guybe7 Dec 15, 2021
ba79b5f
use uppercase flag/categories in jsons + use command flags, not string
guybe7 Dec 15, 2021
faa270b
remove module API
guybe7 Dec 15, 2021
eb968b8
split flags and ACL categories
guybe7 Dec 15, 2021
adb40d0
fix ACL tests
guybe7 Dec 15, 2021
7269e47
add missing information
guybe7 Dec 15, 2021
1036f93
fix complexity
guybe7 Dec 15, 2021
2d6b7fb
final fix
guybe7 Dec 15, 2021
ebd9216
final fix 2
guybe7 Dec 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions runtest-moduleapi
Expand Up @@ -45,3 +45,4 @@ $TCLSH tests/test_helper.tcl \
--single unit/moduleapi/subcommands \
--single unit/moduleapi/reply \
"${@}"

5 changes: 1 addition & 4 deletions src/Makefile
Expand Up @@ -309,7 +309,7 @@ endif

REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX)
REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX)
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o eval.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script_lua.o script.o functions.o function_lua.o
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o eval.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script_lua.o script.o functions.o function_lua.o commands.o
REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX)
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o redisassert.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX)
Expand Down Expand Up @@ -451,9 +451,6 @@ valgrind:
helgrind:
$(MAKE) OPTIMIZATION="-O0" MALLOC="libc" CFLAGS="-D__ATOMIC_VAR_FORCE_SYNC_MACROS" REDIS_CFLAGS="-I/usr/local/include" REDIS_LDFLAGS="-L/usr/local/lib"

src/help.h:
@../utils/generate-command-help.rb > help.h

install: all
@mkdir -p $(INSTALL_BIN)
$(call MAKE_INSTALL,$(REDIS_SERVER_NAME),$(INSTALL_BIN))
Expand Down
54 changes: 28 additions & 26 deletions src/acl.c
Expand Up @@ -61,27 +61,27 @@ struct ACLCategoryItem {
const char *name;
uint64_t flag;
} ACLCommandCategories[] = { /* See redis.conf for details on each category. */
{"keyspace", CMD_CATEGORY_KEYSPACE},
{"read", CMD_CATEGORY_READ},
{"write", CMD_CATEGORY_WRITE},
{"set", CMD_CATEGORY_SET},
{"sortedset", CMD_CATEGORY_SORTEDSET},
{"list", CMD_CATEGORY_LIST},
{"hash", CMD_CATEGORY_HASH},
{"string", CMD_CATEGORY_STRING},
{"bitmap", CMD_CATEGORY_BITMAP},
{"hyperloglog", CMD_CATEGORY_HYPERLOGLOG},
{"geo", CMD_CATEGORY_GEO},
{"stream", CMD_CATEGORY_STREAM},
{"pubsub", CMD_CATEGORY_PUBSUB},
{"admin", CMD_CATEGORY_ADMIN},
{"fast", CMD_CATEGORY_FAST},
{"slow", CMD_CATEGORY_SLOW},
{"blocking", CMD_CATEGORY_BLOCKING},
{"dangerous", CMD_CATEGORY_DANGEROUS},
{"connection", CMD_CATEGORY_CONNECTION},
{"transaction", CMD_CATEGORY_TRANSACTION},
{"scripting", CMD_CATEGORY_SCRIPTING},
{"keyspace", ACL_CATEGORY_KEYSPACE},
{"read", ACL_CATEGORY_READ},
{"write", ACL_CATEGORY_WRITE},
{"set", ACL_CATEGORY_SET},
{"sortedset", ACL_CATEGORY_SORTEDSET},
{"list", ACL_CATEGORY_LIST},
{"hash", ACL_CATEGORY_HASH},
{"string", ACL_CATEGORY_STRING},
{"bitmap", ACL_CATEGORY_BITMAP},
{"hyperloglog", ACL_CATEGORY_HYPERLOGLOG},
{"geo", ACL_CATEGORY_GEO},
{"stream", ACL_CATEGORY_STREAM},
{"pubsub", ACL_CATEGORY_PUBSUB},
{"admin", ACL_CATEGORY_ADMIN},
{"fast", ACL_CATEGORY_FAST},
{"slow", ACL_CATEGORY_SLOW},
{"blocking", ACL_CATEGORY_BLOCKING},
{"dangerous", ACL_CATEGORY_DANGEROUS},
{"connection", ACL_CATEGORY_CONNECTION},
{"transaction", ACL_CATEGORY_TRANSACTION},
{"scripting", ACL_CATEGORY_SCRIPTING},
guybe7 marked this conversation as resolved.
Show resolved Hide resolved
{NULL,0} /* Terminator. */
};

Expand Down Expand Up @@ -436,7 +436,7 @@ void ACLSetUserCommandBitsForCategoryLogic(dict *commands, user *u, uint64_t cfl
while ((de = dictNext(di)) != NULL) {
struct redisCommand *cmd = dictGetVal(de);
if (cmd->flags & CMD_MODULE) continue; /* Ignore modules commands. */
if (cmd->flags & cflag) {
if (cmd->acl_categories & cflag) {
ACLChangeCommandPerm(u,cmd,value);
}
if (cmd->subcommands_dict) {
Expand Down Expand Up @@ -464,7 +464,7 @@ void ACLCountCategoryBitsForCommands(dict *commands, user *u, unsigned long *on,
dictEntry *de;
while ((de = dictNext(di)) != NULL) {
struct redisCommand *cmd = dictGetVal(de);
if (cmd->flags & cflag) {
if (cmd->acl_categories & cflag) {
if (ACLGetUserCommandBit(u,cmd->id))
(*on)++;
else
Expand Down Expand Up @@ -1875,6 +1875,8 @@ void ACLFreeLogEntry(void *leptr) {
*
* The last 2 arguments are a manual override to be used, instead of any of the automatic
* ones which depend on the client and reason arguments (use NULL for default).
*
* If `object` is not NULL, this functions takes over it.
*/
void addACLLogEntry(client *c, int reason, int context, int argpos, sds username, sds object) {
/* Create a new entry. */
Expand All @@ -1885,7 +1887,7 @@ void addACLLogEntry(client *c, int reason, int context, int argpos, sds username
le->ctime = mstime();

if (object) {
le->object = sdsnew(object);
le->object = object;
} else {
switch(reason) {
case ACL_DENIED_CMD: le->object = sdsnew(c->cmd->name); break;
Expand Down Expand Up @@ -2171,7 +2173,7 @@ void aclCommand(client *c) {
while ((de = dictNext(di)) != NULL) {
struct redisCommand *cmd = dictGetVal(de);
if (cmd->flags & CMD_MODULE) continue;
if (cmd->flags & cflag) {
if (cmd->acl_categories & cflag) {
addReplyBulkCString(c,cmd->name);
arraylen++;
}
Expand Down Expand Up @@ -2303,7 +2305,7 @@ void addReplyCommandCategories(client *c, struct redisCommand *cmd) {
int flagcount = 0;
void *flaglen = addReplyDeferredLen(c);
for (int j = 0; ACLCommandCategories[j].flag != 0; j++) {
if (cmd->flags & ACLCommandCategories[j].flag) {
if (cmd->acl_categories & ACLCommandCategories[j].flag) {
addReplyStatusFormat(c, "@%s", ACLCommandCategories[j].name);
flagcount++;
}
Expand Down