Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add contrib/snapshot_leveldb.py #59

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

hltbra
Copy link
Contributor

@hltbra hltbra commented Oct 22, 2020

A copy of snapshot_lmdb with adapations to leveldb. The main difference is
that it uses shutil.copytree() to copy the whole LevelDB directory.

This script is only safe when the dredis server is stopped to guarantee
that all LevelDB in-memory operations made it to the filesystem!


I tested it locally with this:

$ python -m dredis.server --port 6377 --backend leveldb --dir /tmp/dredis-data-test &
[1] 92257

$ 2020-10-22 12:49:31,388 [INFO] Backend: leveldb
2020-10-22 12:49:31,388 [INFO] Port: 6377
2020-10-22 12:49:31,388 [INFO] Root directory: /tmp/dredis-data-test
2020-10-22 12:49:31,388 [INFO] PID: 92257
2020-10-22 12:49:31,388 [INFO] Readonly: false
2020-10-22 12:49:31,388 [INFO] Ready to accept connections


$ redis-cli -p 6377 set test "hello hugo"
OK

$ fg
python -m dredis.server --port 6377 --backend leveldb --dir /tmp/dredis-data-test
^C2020-10-22 12:49:45,847 [INFO] Shutting down...


$ python -m dredis.contrib.snapshot_leveldb --dir /tmp/dredis-data-test --output rdb
2020-10-22 12:50:40,416 [INFO] Copying LevelDB files...
2020-10-22 12:50:40,487 [INFO] Saving RDB file...
2020-10-22 12:50:41,835 [INFO] Done!

$ cp dump*rdb /tmp/dump.rdb

$ cd /tmp

$ redis-server &
[1] 93378
93378:C 22 Oct 2020 12:51:02.242 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
93378:C 22 Oct 2020 12:51:02.242 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=93378, just started
93378:C 22 Oct 2020 12:51:02.242 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 93378
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

93378:M 22 Oct 2020 12:51:02.246 # Server initialized
93378:M 22 Oct 2020 12:51:02.246 * DB loaded from disk: 0.000 seconds
93378:M 22 Oct 2020 12:51:02.246 * Ready to accept connections

$ redis-cli -p 6379 get test
"hello hugo"

$ 

A copy of snapshot_lmdb with adapations to leveldb. The main difference is
that it uses shutil.copytree() to copy the whole LevelDB directory.

This script is only safe when the dredis server is stopped to guarantee
that all LevelDB in-memory operations made it to the filesystem!
The CI started complaining about missing leveldb/db.h when
installing plyvel.
@hltbra hltbra force-pushed the feature/contrib--snapshot-leveldb branch from b2e7953 to 0bfecee Compare October 26, 2020 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant