While open a file for random read: ./home/db/0/001209.sst: Too many open files,reason=Compaction,count=1

[tianguoxin]

使用原生的配置文件
单进程 向1号库中使用mset大量的创建键值对

执行info命令能看到如下错误：

RocksdbBgError

rocksdb_bg_error_count:7
rocksdb0:bgerror=IO error: While open a file for random read: ./home/db/0/001209.sst: Too many open files,reason=Compaction,count=1
rocksdb1:bgerror=IO error: While open a file for random read: ./home/db/1/001220.sst: Too many open files,reason=Compaction,count=1
rocksdb2:bgerror=IO error: While open a file for random read: ./home/db/2/001209.sst: Too many open files,reason=Compaction,count=1
rocksdb4:bgerror=IO error: While open a file for random read: ./home/db/4/001216.sst: Too many open files,reason=Compaction,count=1
rocksdb6:bgerror=IO error: While open a file for random read: ./home/db/6/001225.sst: Too many open files,reason=Compaction,count=1
rocksdb8:bgerror=IO error: While open a file for random read: ./home/db/8/001217.sst: Too many open files,reason=Compaction,count=1
rocksdb9:bgerror=IO error: While open a file for random read: ./home/db/9/001218.sst: Too many open files,reason=Compaction,count=1

在日志中显示如下：
E0625 17:29:39.692903 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 6. It lead to partial success.
E0625 17:29:39.692951 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/2/001209.sst: Too many open files
E0625 17:29:39.692968 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 2. It lead to partial success.
E0625 17:29:39.693002 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/9/001218.sst: Too many open files
E0625 17:29:39.693018 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 9. It lead to partial success.
E0625 17:29:39.714502 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/9/001218.sst: Too many open files
E0625 17:29:39.714542 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 9. It lead to partial success.
E0625 17:29:39.714584 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/2/001209.sst: Too many open files
E0625 17:29:39.714601 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 2. It lead to partial success.
E0625 17:29:39.715062 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/4/001216.sst: Too many open files
E0625 17:29:39.715086 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 4. It lead to partial success.
E0625 17:29:39.715121 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/0/001209.sst: Too many open files
E0625 17:29:39.715137 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 0. It lead to partial success.
E0625 17:29:39.715173 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/8/001217.sst: Too many open files
E0625 17:29:39.715191 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 8. It lead to partial success.
E0625 17:29:39.715263 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/6/001225.sst: Too many open files
E0625 17:29:39.715282 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 6. It lead to partial success.
E0625 17:29:39.715317 29213 rocks_kvstore.cpp:2661] Get unexpected error from rocksdb:IO error: While open a file for random read: ./home/db/1/001220.sst: Too many open files
E0625 17:29:39.715329 29213 session_ctx.cpp:167] mset(nx) commit error at kvstore 1. It lead to partial success.

[takenliu]

这个是说的进程打开文件句柄太多，可以通过下面看看句柄占用情况：
$ ls /proc/$pid/fd
$ lsof -p $pid
客户端连接数大概有多少，redis cluster相关的连接数大概有多少？通过下面命令查看：
redis-client xxx info clients
redis-client xxx client list
另外，rocks.max_open_files配置项默认值为-1，这儿指的是不限制打开sst文件数量。