Skip to content
import and export data from Redis in TSV format
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
spec
src
.gitignore
.travis.yml
LICENSE
Makefile
README.md
shard.yml

README.md

redis-tsv Build Status

import and export data from Redis in TSV format

% redis-tsv export > backup.tsv
% redis-tsv import backup.tsv
% redis-tsv keys > keys.list

Benchmark

  • Bench: import, export and keys with count option (default: 1000)
COUNT Time QPS Unix Command
import 1000 21.9 sec 457567 redis-tsv import -c 1000 10M.tsv
10000 19.6 sec 509422 redis-tsv import -c 10000 10M.tsv
100000 16.2 sec 617951 redis-tsv import -c 100000 10M.tsv
export 1000 465.1 sec 21500 redis-tsv export -c 1000 > 10M.tsv
10000 90.9 sec 110002 redis-tsv export -c 10000 > 10M.tsv
100000 17.2 sec 582429 redis-tsv export -c 100000 > 10M.tsv
keys 1000 10.9 sec 915983 redis-tsv keys -c 1000 > keys.list
10000 8.9 sec 1124260 redis-tsv keys -c 10000 > keys.list
100000 7.6 sec 1311586 redis-tsv keys -c 100000 > keys.list
  • CPU: Intel(R) Core(TM) i7-4800MQ CPU @ 2.70GHz
  • TSV: 10M.tsv (10M entries where key = val = "%08d")
00000001        00000001
00000002        00000002
...
10000000        10000000

Installation

crystal deps  # for the first time only
make
cp bin/redis-tsv ~/bin/

Usage

Bulk operations

  • import
redis-tsv import foo.tsv
redis-tsv -d, import foo.csv
  • export
redis-tsv export > foo.tsv
redis-tsv -d, export > foo.csv
  • keys
redis-tsv keys > keys.list

options

  • -c grows performance, but uses much memory and locks server.
redis-tsv export -c 1000  > foo.tsv  # 1m13s (default)
redis-tsv export -c 10000 > foo.tsv  # 7m28s (6 times faster)

(entry count: 10M)

information

util commands for easy access to INFO

  • count : show a number of keys
% redis-tsv count
10000000
  • version : show the redis version
% redis-tsv version
3.0.6
  • role : show the role of replication
% redis-tsv role
master
  • ping : execute PING command
% redis-tsv ping
PONG
  • info (INFO result itself)
% redis-tsv info
# Server
redis_version:3.0.6
redis_git_sha1:00000000
...

Test

  • CAUTION: This spec flushes your redis db on localhost:6379.
% crystal spec

TODO

v0.3.0

  • import to redis-cluster

v0.4.0

  • keys from redis-cluster

v0.5.0

  • export from redis-cluster

Contributing

  1. Fork it ( https://github.com/maiha/redis-tsv.cr/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

  • maiha maiha - creator, maintainer
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.