-
Notifications
You must be signed in to change notification settings - Fork 536
/
RELEASE-NOTES
249 lines (222 loc) · 12.8 KB
/
RELEASE-NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
-------------------------
Riak 1.0.0 Release Notes
-------------------------
NOTE: These release notes should be considered a work in progress
until the final 1.0.0 release is made. Bug fixes will be added,
but major features should, in all likelihood, be represented below.
Major Features and Improvements for Riak 1.0.0
----------------------------------------------
1) 2i
Secondary Indexes (2I) makes it easier to find your data in
Riak. With 2i, an application can index an object under an arbitrary
number of field/value pairs specified at write time. Later, the
application can query the index, pulling back a list of keys that
match a given value or fall within a given range of values. This
simple mechanism enables applications to model alternate keys,
one-to-many relationships, and many-to-many relationships in
Riak. Indexing is atomic and real-time. Querying is integrated with
MapReduce. In this release, the query feature set is minimal: only
exact match and range queries are supported, and only on a single
field. In addition, the system must be configured to use the
riak_kv_eleveldb_backend. Future releases will bring support for more
complex queries and additional backends, depending upon user feedback.
2) Backend Refactoring
There are 3 main areas of change with regard to the backend refactoring.
* Changes to the suite of available backends
* Added: eleveldb backend
* Merged: ets and cache backend have been consolidated into a single
memory backend.
* Removed: dets, filesystem (fs), and gb_trees backends
* Standardized API
* All of the backend modules now share a common API.
* Vnode code has been greatly simplified.
* Asynchronous folding
* The vnode no longer has to be blocked by fold operations.
* A node for a backend capable of asynchronous folds starts a worker
pool to handle fold requests.
* Asynchronous folds are done using read-only snapshots of the data.
* Bitcask, eleveldb, memory, and multi backends support asynchronous folding.
* Asynchronous folding may be disabled by adding {async_folds, false}
in the riak_kv section of the app.config file.
3) Lager
Lager is Riak’s new logging library. It aims to alleviate some of the
shortcomings in Erlang/OTP’s bundled logging library; specifically
more readable log messages, support for external logfile rotation,
more granular log levels and runtime changes.
4) LevelDB
Riak now has support for LevelDB, a fast key-value storage library
written at Google that provides an ordered mapping from string keys to
string values. This backend provides a viable alternative to the
innostore backend, with better performance. LevelDB has an append-only
design that provides high-performance writes. Unlike Bitcask, however,
LevelDB keeps keys in sorted order and uses a multi-level cache to
keep portions of the keyspace in memory. This design allows LevelDB to
effectively deal with very large keyspaces, but at the cost of
additional seeks on read.
5) Pipe
Riak Pipe is the new plumbing for Riak's MapReduce system. It improves
upon the legacy system in many ways, including built-in backpressure
to keep phases from overloading others downstream, and a design shaped
to make much better use of the parallel-processing resources available
in the cluster. Riak Pipe is also a general queue-based parallel
processing system, which new applications may find is a more natural
starting point than bare Riak Core. Riak Pipe is the default plumbing
for MapReduce queries on new clusters. To move old clusters onto Riak
Pipe MapReduce, add `{mapred_system, pipe}` to the `riak_kv` section
of your node’s app.config. To force Riak to use the legacy system for
MapReduce, set `{mapred_system, legacy}` in the `riak_kv` section of
app.config instead.
6) URL encoding changes over REST API
Riak now decode URL encoded bucket/keys over the REST API, rather than
prior behavior which was to decode links but not decode
buckets/keys. The default cluster behavior is configurable in
riak_core section of app.config: {http_url_decoding, on} provides the
new behavior (decode everything), missing/anything else provides the
current behavior.
The module riak_kv_encoding_migrate.erl is also provided to help
migrate existing encoded buckets/keys to their decoded equivalents.
7) Riak Clustering Overhaul
The clustering subsystem of Riak has been overhauled for 1.0, addressing a
variety of bugs and limitations.
* When reassigning a partition to a new node, the partition data from
the existing owner is first transferred to the new owner before
officially changing the ownership in the ring. This fixes a bug
where 404s could appear while ownership was being changed.
* Adding/removing nodes to a cluster is now more robust, and it is no
longer necessary to check for ring convergence (riak-admin ringready)
between adding/removing nodes. Adding multiple nodes all at once should
"just work".
* Handoff related to changing node owners can now occur while a cluster is
under load, therefore allowing a Riak cluster to scale up and down during
load / normal operation.
* Various other clustering bug/fixes. See the fixed bug list for details.
Notes:
* "riak-admin join" has new semantics. The command is now a one-way
operation that joins a single node to cluster. The node that the
command is executed under should be the desired joining node, and
the target of the command should be a member of the desired target
cluster. The new command requires the joining node to be a singleton
(1-node) cluster.
* "riak-admin leave" is now the only safe way to remove a node from a
cluster. The leave command ensures that the exiting node will
handoff all its partitions before leaving the cluster. It should be
executed by the node intended to leave.
* "riak-admin remove" is now changed to a force-remove, where a node
is immediately removed from the cluster without waiting on
handoff. This is designed for cases where a node is unrecoverable
and for which handoff does not make sense.
* The new cluster changes require all nodes to be up and reachable in
order for new members to be integrated into the cluster and for the
data to be rebalanced. During brief node outages, the new protocol
will wait until all nodes are eventually back online and continue
automatically. If it is known that a node will be offline for an
extended period, the new 'riak-admin down' command can be used to
mark a node as offline and the cluster will then resume integrating
nodes and performing ring rebalances. Nodes marked as down will
automatically rejoin and reintegrate into the cluster when they come
back online.
8) Get/Put/Delete Improvements
The way that Riak versions and updates objects has been
overhauled. ClientIds are no longer used when updating objects, the
server handles all versioning using a vector clock id per-vnode.
New clusters are configured with the new vclock behavior turned on.
If you are performing a rolling upgrade of an existing cluster, once
all nodes have been upgraded the app.config needs to be updated to add
{vnode_vclocks, true}.
Puts are now coordinated in the same way as on the original Dynamo
system. Requests must be handled by a node in the preference list
(primary or fallback) for that bucket/key. Nodes will automatically
forward to a valid node when necessary and increment the coord_redirs
stats counter. The put is initially written to the local vnode before
forwarding to the remote vnodes. This ensures that the updated vclock
for the riak object will replace the existing value or create siblings
in partitioning/failure scenario where the same client can see both
sides.
Error proofing for the failure scenarios has made it so that clients
no longer have to be well behaved. If allow_mult is set true, every
time you create a new object and put over an existing one it will
create a sibling. Vector clocks should now be much smaller in size as
only a few vclock ids are now updated. This should resolve a number of
issues due to vclock pruning causing siblings.
Gets have been changed to return more information during failure.
Prior to 1.0 there were cases where Riak returned not found if not
enough valid responses were returned. The case of not enough responses
has been changed to an error instead reported as 503 over HTTP or as
{error, {r_val_unsatistfied, R, NumResponses}} for Erlang/PBC clients.
New options have been added to the get requests for handling
notfound. Prior to 1.0 only successful reads were counted towards R
and there was some logic to try and fail early rather than wait until
the request timed out if not enough replies were received
(basic_quorum). This meant when a node went down and another node
didn’t response you would get a not found response that triggered a
read repair and then if you retrieved the object again it would be
present.
Now that other enhancements have been made (delete and asynchronous
improvements to the vnodes) we can change notfounds to be counted
towards R and disable the basic_quorum logic by setting bucket
properties to [{notfound_ok, true}, {basic_quorum, false}] and reduce
the number of cases where notfound is returned on first request when
an object could be.
Bugs Fixed
----------
bz0105 - Python client new_binary doesn't set the content_type well
bz0123 - default_bucket_props in app.config is not merged with the hardcoded defaults
bz0218 - bin/riak-admin leave needs to remove abandoned ring files
bz0260 - Expose tombstones as conflicts when allow_mult is true
bz0294 - Possible race condition in nodetool
bz0325 - Patch for mapred_builtins.js - reduceMin and reduceMax
bz0420 - Links are incorrectly translated in riak_object:dejsonify_values/2
bz0426 - bin/riak-admin leave has poor console output
bz0441 - detect and report bad datafile entry
bz0461 - Guard against non-string values of content-type in riak-erlang-client
bz0502 - Minor merge_index code cleanup
bz0564 - Planner's subprocesses run long after {timeout, range_loop} exception
bz0599 - Consider adding erlang:memory/0 information to stats output
bz0605 - riak_kv_wm_raw does not handle put_fsm timeout
bz0617 - Riak URL decodes keys submitted in the Link header
bz0688 - Ring does not settle when building large clusters
bz0710 - "riak ping" exits with status 0 when ping fails
bz0716 - Handoff Sender crashes loudly when remote node dies
bz0808 - The use of fold/3 function in do_list_keys/6 in riak_kv_vnode
does not allow backends to take advantage of bucket aware
optimizations
bz0823 - Handoff processes crash irretrievably when receiving TCP garbage,
resulting in node failure
bz0869 - Any commands that change the ring should require the ringready command to
return TRUE
bz0878 - riak-admin leave, then stop node, then restart -> handoff transfers do not resume
bz0911 - Fix #scope{} and #group{} operator preplanning
bz0931 - Cluster should not use partition ownership to find list of nodes
bz0939 - Fast map phase can overrun slower reduce phase
bz0953 - Change Riak Search to use the Whitespace analyzer by default
bz0954 - Wildcard queries are broken with Whitespace analyzer
bz0967 - Upgrade riak_search to compile on Erlang R14B01
bz0989 - riak_kv_map_master crashes when counters are out of date
bz1003 - REST API and PBC API have incompatible naming rules
bz1024 - Valid objects return notfound during heavy partition transfer
bz1033 - delete_resource doesn't handle case where object is no longer extant
bz1047 - Javascript VM worker process is not restarted after crash
bz1052 - riak_core_ring_handler:ensure_vnodes_started breaks on multiple vnode types
bz1055 - riak_core_vnode_master keeps unnecessary "exclusions list" in its state
bz1067 - deprecate riak_kv_util:[try_cast/fallback]
bz1072 - spiraltime crash (in riak_kv_stat)
bz1075 - java.net.SocketException: Connection reset by peer from proto client
(thundering (small) herd)
bz1077 - nodetool needs to support Erlang SSL distribution
bz1097 - Truncated data file then merge triggers error in bitcask_fileops:fold/3
bz1109 - PB interface error when content-type is JSON and {not_found} in results
bz1110 - Riak Search integration with MapReduce does not work as of Riak Search 0.14.2rc9
bz1125 - HTTP Delete returns a 204 when the RW param cannot be satisfied, expected 500
bz1126 - riak_kv_cache_backend doesn't stop
bz1144 - Riak Search custom JS extractor not initializing VM pool properly
bz1147 - "Proxy Objects" are not cleaned
bz1149 - Delete op should not use user-supplied timeout for tombstone harvest
bz1155 - Regression in single negated term
bz1165 - mi_buffer doesn’t check length when reading terms
bz1175 - Riak_kv_pb_socket crashes when clientId is undefined
bz1176 - Error on HTTP POST or PUT that specifies indexes with integer values > 255
and returnbody=true.
bz1177 - riak_kv_bitcask_backend.erl's use of symlinks breaks upgrade from 0.14.2
bz1178 - ring mgr and bucket fixups not playing well on startup
All bug and issue numbers reference https://issues.basho.com
Enjoy 1.OMG!