-
Notifications
You must be signed in to change notification settings - Fork 27
dbmon.sh
dbmon.sh is a tool used to monitor database and entry cache usage. It is especially useful for database cache and entry/dn cache tuning - how much space is left, is the cache full, how much space on average do I need per entry/dn.
- bash
- openldap clients - the openldap ldapsearch command
- awk - GNU awk
Please refer to https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Performance_Tuning_Guide/memoryusage.html for an explanation of cache sizing.
It doesn't take any command line arguments, so all options must be passed in as environment variables
INCR=1 HOST=ldap.example.com BINDDN="cn=directory manager" BINDPW="secret" VERBOSE=2 dbmon.sh
dbmon.sh will loop repeatedly showing the db information until it is killed or Ctrl-C
All arguments are optional, but you will most likely have to provide BINDPW
- INCR - show results every INCR seconds - default is 1 second
- HOST - name of host or IP address - default is "localhost"
- PORT - port number (LDAP not LDAPS) - default is 389
- BINDDN - DN to use to bind - must have permission to read everything under cn=config - default is cn=Directory Manager
- BINDPW - password for BINDDN - default is secret
- DBLIST - a list of databases you want to check - default is "all"
-
for more than one, delimit with spaces e.g. DBLIST="one two three" - INDEXLIST - a list of indexes to show for each named database - default is none
-
specify "all" for all indexes, or named e.g. INDEXLIST="id2entry entryrdn" - VERBOSE - output level - 0 == suitable for parsing by a script - 1 == has column headings - 2 == provides detailed descriptions of the data - default is 0
Here is some sample output:
$ BINDPW=password VERBOSE=2 ~/scripts/dbmon.sh
Tue Oct 28 17:34:45 MDT 2014
# dbcachefree - free bytes in dbcache
# free% - percent free in dbcache
# roevicts - number of read-only pages dropped from cache to make room for other pages
# if this is non-zero, it means the dbcache is maxed out and there is page churn
# hit% - percent of requests that are served by cache
# pagein - number of pages read into the cache
# pageout - number of pages dropped from the cache
dbcachefree 397310 free% 2.2 roevicts 9282348 hit% 50 pagein 2934772 pageout 219075
# dbname - name of database instance - the row shows the entry cache stats
# count - number of entries in cache
# free - number of free bytes in cache
# free% - percent free in cache
# size - average size of date in cache in bytes (current size/count)
# DNcache - the line below the entry cache stats are the DN cache stats
# count - number of dns in dn cache
# free - number of free bytes in dn cache
# free% - percent free in dn cache
# size - average size of dn in dn cache in bytes (currentdncachesize/currentdncachecount)
# under each db are the list of selected indexes specified with INDEXLIST
dbname count free free% size
userroot:ent 50000 2400 0.8 8972.7
userroot:dn 100000 4294735 69.8 130.0
dbscan -f /var/lib/dirsrv/slapd-inst/db/userRoot/id2entry.db -t 200|grep -c rdn:
If your caches are big enough, this number should match the count column above. If not, see below.
In this case, there are 100000 (count) DNs in the DN cache (userroot:dn). There is ~4MB free space (4294735) in bytes, and the percentage of free bytes in the DN cache is 68%. Based on the count and the bytes used, each DN in memory takes up about 130 bytes (130.0). This looks pretty good - the DN cache has plenty of room.
However, the entry cache (userroot:ent) does not look so good. There are only 50000 entries, only half, and the free bytes is 2400, not nearly enough to cache another entry (of average size 8972.7 bytes). In order to cache all of the entries, you will need at a minimum 100000 * 8972.7 = 897270000 bytes in the entry cache. And this will leave no room to grow, which is especially important for replication (for the state metadata). One rule of thumb is to round up to the nearest GB (e.g. 1GB) and multiply by 2, giving an estimated starting cache size of 2GB (2147483648 bytes). If you have enough RAM, on a 64-bit system, you could give yourself much more room and go for 8GB or higher. To change the entry cache size:
ldapmodify -x -D "cn=directory manager" -w password <<EOF
dn: cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsslapd-cachememsize
nsslapd-cachememsize: 2147483648
EOF
A restart is required:
# service dirsrv restart
OR
# systemctl restart dirsrv.target
NOTE: userRoot is the default user/group database name. Yours may be different. dbmon.sh by default will give you the report for all databases.
dbcachefree 397310 free% 2.2 roevicts 9282348 hit% 50 pagein 2934772 pageout 219075
In this case, the DB cache has only 2.2% free. The DB needs at least %15 free to operate efficiently. The high number for roevicts indicates that there is a lot of page churn. The size of the DB cache needed to cache all database pages in memory is roughly the same as the size of all of your database files and indexes (e.g. /var/lib/dirsrv/slapd-inst/db//.db[4]) + about 12% for overhead.
ldapmodify -x -D "cn=directory manager" -w password <<EOF
dn: cn=config,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsslapd-dbcachesize
nsslapd-dbcachesize: 2147483648
EOF
A restart is required:
# service dirsrv restart
OR
# systemctl restart dirsrv.target
By default, the directory server uses memory mapped files for the DB cache and metadata (/var/lib/dirsrv/slapd-inst/db/__db.*). If you have a lot of RAM, you can also use a RAM disk. The easiest way is to
mkdir -p /dev/shm/slapd-inst
ldapmodify -x -D "cn=directory manager" -w password <<EOF
dn: cn=config,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsslapd-db-home-directory
nsslapd-db-home-directory: /dev/shm/slapd-inst
EOF
Then restart the server as above.
WARNING: This will make the cache non-persistent. That means, if you have to reboot the machine, your cache will have to be rebuilt. This should cause the server to take a bit longer to startup, and initial operations a bit slower until the cache is warmed up.
If you don't have enough RAM to cache everything in the entry cache, the DN cache, and the DB cache, make sure you max out the entry cache and DN caches first. Those have the greatest impact on performance. Then, use whatever you have remaining for the DB cache.
Then, use dbmon.sh to periodically monitor cache utilization to see if more is needed.