Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100755 137 lines (108 sloc) 3.507 kb
b4783f0 Srini Kommoori 0.1.0 commit
authored
1 h2. leveldb-server
a1af470 Srini Kommoori 0.1.0 commit
authored
2
f879da0 Srini Kommoori 0.1.0 commit
authored
3 * Async leveldb server and client based on zeromq
4 * Storage engine *"leveldb":http://code.google.com/p/leveldb/*
5 * Networking library *"zeromq":http://www.zeromq.org/*
6 * We use leveldb-server at *"Safebox":http://safebox.fabulasolutions.com/*
b4783f0 Srini Kommoori 0.1.0 commit
authored
7
8 h3. License
a1af470 Srini Kommoori 0.1.0 commit
authored
9
f879da0 Srini Kommoori 0.1.0 commit
authored
10 New BSD license. Please see license.txt for more details.
b4783f0 Srini Kommoori 0.1.0 commit
authored
11
12 h2. Features
13
14 * Very simple key-value storage
a1af470 Srini Kommoori 0.1.0 commit
authored
15 * Data is sorted by key - allows @range@ queries
b4783f0 Srini Kommoori 0.1.0 commit
authored
16 * Data is automatically compressed
5479535 Srini Kommoori 0.1.0 commit
authored
17 * Can act as persistent cache
384a9ed Srini Kommoori 0.1.0 commit
authored
18 * For our use at *"Safebox":http://safebox.fabulasolutions.com/* it replaced memcached+mysql
aa9eccf Srini Kommoori 0.1.0 commit
authored
19 * Simple backups @cp -rf level.db backup.db@
5479535 Srini Kommoori 0.1.0 commit
authored
20 * Networking/wiring from @zeromq@ messaging library - allows many topologies
b4783f0 Srini Kommoori 0.1.0 commit
authored
21 * Async server for scalability and capacity
22 * Sync client for easy coding
aa9eccf Srini Kommoori 0.1.0 commit
authored
23 * Easy polyglot client bindings. See *"zmq bindings":http://www.zeromq.org/bindings:_start*
b4783f0 Srini Kommoori 0.1.0 commit
authored
24
25 <pre>
26 >>> db.put("k3", "v3")
27 'True'
28 >>> db.get("k3")
29 'v3'
30 >>> db.range()
31 '[{"k1": "v1"}, {"k2": "v2"}, {"k3": "v3"}]'
6954564 Srini Kommoori 0.1.0 commit
authored
32 >>> db.range("k1", "k2")
33 '[{"k1": "v1"}, {"k2": "v2"}]'
b4783f0 Srini Kommoori 0.1.0 commit
authored
34 >>> db.delete('k1')
35 >>>
36 </pre>
37
38 Will be adding high availability, replication and autosharding using the same zeromq framework.
39
40 h3. Dependencies
6954564 Srini Kommoori 0.1.0 commit
authored
41
b4783f0 Srini Kommoori 0.1.0 commit
authored
42 <pre>
aa9eccf Srini Kommoori 0.1.0 commit
authored
43 python 2.6+ (older versions with simplejson)
b4783f0 Srini Kommoori 0.1.0 commit
authored
44 zmq
45 pyzmq
46 leveldb
47 pyleveldb
48 </pre>
49
50 h2. Getting Started
51
f879da0 Srini Kommoori 0.1.0 commit
authored
52 Instructions for an EC2 Ubuntu box.
53
b4783f0 Srini Kommoori 0.1.0 commit
authored
54 h3. Installing zeromq
55
56 <pre>
57 wget http://download.zeromq.org/zeromq-2.1.10.tar.gz
58 tar xvfz zeromq-2.1.10.tar.gz
59 cd zeromq-2.1.10
60 sudo ./configure
61 sudo make
62 sudo make install
63 </pre>
64
65 h3. Installing pyzmq
a1af470 Srini Kommoori 0.1.0 commit
authored
66
b4783f0 Srini Kommoori 0.1.0 commit
authored
67 <pre>
68 wget https://github.com/zeromq/pyzmq/downloads/pyzmq-2.1.10.tar.gz
69 tar xvfz pyzmq-2.1.10.tar.gz
70 cd pyzmq-2.1.10/
71 sudo python setup.py configure --zmq=/usr/local/lib/
72 sudo python setup.py install
73 </pre>
74
75 h3. Installing leveldb and pyleveldb
a1af470 Srini Kommoori 0.1.0 commit
authored
76
b4783f0 Srini Kommoori 0.1.0 commit
authored
77 <pre>
78 svn checkout http://py-leveldb.googlecode.com/svn/trunk/ py-leveldb-read-only
79 cd py-leveldb-read-only
80 sudo compile_leveldb.sh
81 sudo python setup.py install
82 </pre>
83
5479535 Srini Kommoori 0.1.0 commit
authored
84 h3. Starting the "leveldb-server":https://github.com/srinikom/leveldb-server/blob/master/leveldb-server.py
a1af470 Srini Kommoori 0.1.0 commit
authored
85
b4783f0 Srini Kommoori 0.1.0 commit
authored
86 <pre>
87 > python leveldb-server.py -h
88 Usage: leveldb-server.py
89 -p [port and host settings] Default: tcp://127.0.0.1:5147
90 -f [database file name] Default: level.db
91
92 leveldb-server
93
94 Options:
95 --version show program's version number and exit
96 -h, --help show this help message and exit
97 -p HOST, --host=HOST
98 -d DBFILE, --dbfile=DBFILE
99 > python leveldb-server.py
100 </pre>
101
5479535 Srini Kommoori 0.1.0 commit
authored
102 h3. Using the "leveldb-client-py":https://github.com/srinikom/leveldb-server/blob/master/clients/py/leveldbClient/database.py
a1af470 Srini Kommoori 0.1.0 commit
authored
103
b4783f0 Srini Kommoori 0.1.0 commit
authored
104 <pre>
aa9eccf Srini Kommoori 0.1.0 commit
authored
105 > cd clients/py/
b4783f0 Srini Kommoori 0.1.0 commit
authored
106 > sudo python setup.py install
107 > python
108 >>> from leveldbClient import database
109 >>> db = database.leveldb()
110 >>> db.get("Key")
111 >>> db.put("K", "V")
112 >>> db.range()
113 >>> db.range(start, end)
114 >>> db.delete("K")
115 </pre>
116
5479535 Srini Kommoori 0.1.0 commit
authored
117 h2. Backups
b4783f0 Srini Kommoori 0.1.0 commit
authored
118 <pre>
119 cp -rpf level.db backup.db
120 </pre>
121
f879da0 Srini Kommoori 0.1.0 commit
authored
122 h2. Known issues and work in progress
123
124 Would love your pull requests on
125 * Benchmarking and performance analysis
126 * client libraries for other languages
127 * [issue] zeromq performance issues with 1M+ inserts at a time
5479535 Srini Kommoori 0.1.0 commit
authored
128 * [feature] timeouts in client library
129 * [feature] support for counters
9e32756 Srini Kommoori 0.1.0 commit
authored
130 * [feature] limit support in range queries
f879da0 Srini Kommoori 0.1.0 commit
authored
131 * Serializing and seperate threads for get/put/range in leveldb-server
132 * HA/replication/autosharding and possibly pub-sub for replication
133
9f6579f Srini Kommoori 0.1.0 commit
authored
134 h2. Thanks
b4783f0 Srini Kommoori 0.1.0 commit
authored
135
9f6579f Srini Kommoori 0.1.0 commit
authored
136 Thanks to all the folks who have contributed to all the dependencies. Special thanks to pyzmq/examples/mongo* author for inspiration.
Something went wrong with that request. Please try again.