-
Notifications
You must be signed in to change notification settings - Fork 831
/
CHANGELOG
12680 lines (8319 loc) 路 488 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
devel
-----
* added "random" masking to mask any data type, added wildcard masking
* report run-time openssl version (for dynamically linked executables)
* added greeting warning about maintainer mode
* improve insertion time into non-unique secondary indexes with the RocksDB
engine
* fixed possible segfault when using COLLECT with a LIMIT and an offset
* fixed COLLECT forgetting top-level variables after 1000 rows
* added sort-limit optimization in AQL; improves memory usage and execution
time for some queries
* upgraded to OpenSSL 1.1.0j
* added configurable masking of dumped data via `arangodump` tool to obfuscate exported sensible data
* fixed arangoimp script for MacOSX CLI Bundle
* added "peakMemoryUsage" in query results figures, showing the peak memory
usage of the executed query. In a cluster, the value the peak memory usage
of all shards, but it is not summed up across shards.
* fixed an issue where a crashed coordinator can lead to some Foxx queue jobs
erroneously either left hanging or being restarted
* fix issue #7900: Bind values of `null` are not replaced by
empty string anymore, when toggling between json and table
view in the web-ui.
* fix issue #7903: Regression on ISO8601 string compatibility in AQL
millisecond parts of AQL date values were limited to up to 3 digits.
Now the length of the millisecond part is unrestricted, but the
millisecond precision is still limited to up to 3 digits.
* fix issue #7900: Bind values of `null` are not replaced by empty string
anymore, when toggling between JSON and table view in the web UI
* the RocksDB primary index can now be used by the optimizer to optimize queries
that use `_key` or `_id` for sorting or for range queries.
* the web UI will now by default show the documents of a collection lexicographically
sorted when sorting documents by their `_key` values.
Previous versions of ArangoDB tried to interpret `_key` values as numeric values if
possible and sorted by these. That previous sort strategy never used an index and
could have caused unnecessary overhead. The new version will now use an index for
sorting for the RocksDB engine, but may change the order in which documents are
shown in the web UI (e.g. now a `_key` value of "10" will be shown before a `_key`
value of "9").
* fixed known issue #445: ArangoSearch ignores `_id` attribute even if `includeAllFields`
is set to `true`.
* upgraded bundled boost library to version 1.69.0
* upgraded bundled curl library to version 7.63
* fix issue #7900: Bind values of `null` are not replaced by empty string anymore,
when toggling between JSON and table view in the web UI.
* Use base64url to encode and decode JWT parts.
* Added --server.jwt-secret-keyfile option.
* speed up data-modification operations in exclusive transactions in the RocksDB
storage engine
* An AQL query that uses the edge index only and returns the opposite side of
the edge can now be executed in a more optimized way, e.g.
FOR edge IN edgeCollection FILTER edge._from == "v/1" RETURN edge._to
is fully covered by RocksDB edge index. For MMFiles this rule does not apply.
* reverted accidental change to error handling in geo index
In previous versions, if non-valid geo coordinates were contained in the
indexed field of a document, the document was simply ignored an not indexed.
In 3.4.0, this was accidentally changed to generate an error, which caused
the upgrade procedure to break in some cases.
* fixed TypeError being thrown instead of validation errors when Foxx manifest
validation fails
* fix internal issue #2786: improved confirmation dialog when clicking the
Truncate button in the Web UI
* make `--help-all` now also show all hidden program options
Previously hidden program options were only returned when invoking arangod or
a client tool with the cryptic `--help-.` option. Now `--help-all` simply
retuns them as well.
The program options JSON description returned by `--dump-options` was also
improved as follows:
- the new boolean attribute "dynamic" indicates whether the option has a dynamic
default value, i.e. a value that depends on the target host capabilities or
configuration
- the new boolean attribute "requiresValue" indicates whether a boolean option
requires a value of "true" or "false" when specified. If "requiresValue" is
false, then the option can be specified without a boolean value following it,
and the option will still be set to true, e.g. `--server.authentication` is
identical to `--server.authentication true`.
- the new "category" attribute will contain a value of "command" for command-like
options, such as `--version`, `--dump-options`, `--dump-dependencies` etc.,
and "option" for all others.
* fixed issue #7586: a running query within the user interface was not shown
if the active view was `Running Queries` or `Slow Query History`.
* fixed issue #7743: Query processing discrepancy between Rocks and MMFiles databases
this change enforces the invalidation of variables in AQL queries after usage of
a COLLECT statement as documented. The documentation for variable invalidation claims
that
The COLLECT statement will eliminate all local variables in the current scope.
After COLLECT only the variables introduced by COLLECT itself are available.
However, the described behavior was not enforced when a COLLECT was preceded by a
FOR loop that was itself preceded by a COLLECT. In the following query the final
RETURN statement accesses variable `key1` though the variable should have been
invalidated by the COLLECT directly before it:
FOR x1 IN 1..2
COLLECT key1 = x1
FOR x2 IN 1..2
COLLECT key2 = x2
RETURN [key2, key1]
In previous releases, this query was
parsed ok, but the contents of variable `key1` in the final RETURN statement were
undefined.
This change is about making queries as the above fail with a parse error, as an
unknown variable `key1` is accessed here, avoiding the undefined behavior. This is
also in line with what the documentation states about variable invalidation.
* fixed issue #7763: Collect after update does not execute updates
* fixed issue #7749: AQL query result changed for COLLECT used on empty data/array
* fixed issue #7757: Using multiple filters on nested objects produces wrong results
* fixed a rare thread local dead lock situation in replication:
If a follower tries to get in sync in the last steps it requires
a lock on the leader. If the follower cancels the lock before the leader
has succeeded with locking we can end up with one thread being deadlocked.
* allow usage of floating point values in AQL without leading zeros, e.g.
`.1234`. Previous versions of ArangoDB required a leading zero in front of
the decimal separator, i.e `0.1234`.
* Foxx `req.makeAbsolute` now will return meaningful values when ArangoDB is using
a unix socket endpoint. URLs generated when using a unix socket follow the format
http://unix:<socket-path>:<url-path> used by other JS tooling.
* Updated joi library (Web UI), improved foxx mount path validation
* do not create `_routing` collection for new installations/new databases,
as it is not needed anymore. Redirects to the web interface's login screen, which
were previously handled by entries in the `_routing` collection are now handled
from the responsible REST action handler directly
Existing `_routing` collections will not be touched as they may contain other
entries as well, and will continue to work.
* do not create `_modules` collection for new databases/installations
`_modules` is only needed for custom modules, and in case a custom
module is defined via `defineModule`, the _modules collection will
be created lazily automatically.
Existing modules in existing `_modules` collections will remain
functional even after this change
* disable in-memory cache for edge and traversal data on agency nodes, as it
is not needed there
* removed bundled Valgrind headers, removed JavaScript variable `valgrind`
from the `internal` module
* upgraded JEMalloc version to 5.1.0
* use `-std=c++14` for ArangoDB compilation
v3.4.1 (XXXX-XX-XX)
-------------------
* fix thread shutdown in _WIN32 builds
Previous versions used a wrong comparison logic to determine the current
thread id when shutting down a thread, leading to threads hanging in their
destructors on thread shutdown
* reverted accidental change to error handling in geo index
In previous versions, if non-valid geo coordinates were contained in the
indexed field of a document, the document was simply ignored an not indexed.
In 3.4.0, this was accidentally changed to generate an error, which caused
the upgrade procedure to break in some cases.
* fixed TypeError being thrown instead of validation errors when Foxx manifest
validation fails
* make AQL REMOVE operations use less memory with the RocksDB storage engine
the previous implementation of batch removals read everything to remove into
memory first before carrying out the first remove operation. The new version
will only read in about 1000 documents each time and then remove these. Queries
such as
FOR doc IN collection FILTER ... REMOVE doc IN collection
will benefit from this change in terms of memory usage.
* make `--help-all` now also show all hidden program options
Previously hidden program options were only returned when invoking arangod or
a client tool with the cryptic `--help-.` option. Now `--help-all` simply
retuns them as well.
The program options JSON description returned by `--dump-options` was also
improved as follows:
- the new boolean attribute "dynamic" indicates whether the option has a dynamic
default value, i.e. a value that depends on the target host capabilities or
configuration
- the new boolean attribute "requiresValue" indicates whether a boolean option
requires a value of "true" or "false" when specified. If "requiresValue" is
false, then the option can be specified without a boolean value following it,
and the option will still be set to true, e.g. `--server.authentication` is
identical to `--server.authentication true`.
- the new "category" attribute will contain a value of "command" for command-like
options, such as `--version`, `--dump-options`, `--dump-dependencies` etc.,
and "option" for all others.
v3.4.0 (2018-12-06)
-------------------
* Add license key checking to enterprise version in Docker containers.
v3.4.0-rc.5 (2018-11-29)
------------------------
* Persist and check default language (locale) selection.
Previously we would not check if the language (`--default-language`) had changed
when the server was restarted. This could cause issues with indexes over text fields,
as it will resulted in undefined behavior within RocksDB (potentially missing entries,
corruption, etc.). Now if the language is changed, ArangoDB will print out an error
message on startup and abort.
* fixed issue #7522: FILTER logic totally broke for my query in 3.4-rc4
* export version and storage engine in `_admin/cluster/health` for Coordinators
and DBServers.
* restrict the total amount of data to build up in all in-memory RocksDB write buffers
by default to a certain fraction of the available physical RAM. This helps restricting
memory usage for the arangod process, but may have an effect on the RocksDB storage
engine's write performance.
In ArangoDB 3.3 the governing configuration option `--rocksdb.total-write-buffer-size`
had a default value of `0`, which meant that the memory usage was not limited. ArangoDB
3.4 now changes the default value to about 50% of available physical RAM, and 512MiB
for setups with less than 4GiB of RAM.
* lower default value for `--cache.size` startup option from about 30% of physical RAM to
about 25% percent of physical RAM.
* fix internal issue #2786: improved confirmation dialog when clicking the truncate
button in the web UI
* Updated joi library (web UI), improved Foxx mount path validation
* disable startup warning for Linux kernel variable `vm.overcommit_memory` settings
values of 0 or 1.
Effectively `overcommit_memory` settings value of 0 or 1 fix two memory-allocation
related issues with the default memory allocator used in ArangoDB release builds on
64bit Linux.
The issues will remain when running with an `overcommit_memory` settings value of 2,
so this is now discouraged.
Setting `overcommit_memory` to 0 or 1 (0 is the Linux kernel's default) fixes issues
with increasing numbers of memory mappings for the arangod process (which may lead
to an out-of-memory situation if the kernel's maximum number of mappings threshold
is hit) and an increasing amount of memory that the kernel counts as "committed".
With an `overcommit_memory` setting of 0 or 1, an arangod process may either be
killed by the kernel's OOM killer or will die with a segfault when accessing memory
it has allocated before but the kernel could not provide later on. This is still
more acceptable than the kernel not providing any more memory to the process when
there is still physical memory left, which may have occurred with an `overcommit_memory`
setting of 2 after the arangod process had done lots of allocations.
In summary, the recommendation for the `overcommit_memory` setting is now to set it
to 0 or 1 (0 is kernel default) and not use 2.
* fixed Foxx complaining about valid `$schema` value in manifest.json
* fix for supervision, which started failing servers using old transient store
* fixed a bug where indexes are used in the cluster while still being
built on the db servers
* fix move leader shard: wait until all but the old leader are in sync.
This fixes some unstable tests.
* cluster health features more elaborate agent records
* agency's supervision edited for advertised endpoints
v3.4.0-rc.4 (2018-11-04)
------------------------
* fixed Foxx queues not retrying jobs with infinite `maxFailures`
* increase AQL query string parsing performance for queries with many (100K+) string
values contained in the query string
* increase timeouts for inter-node communication in the cluster
* fixed undefined behavior in `/_api/import` when importing a single document went
wrong
* replication bugfixes
* stop printing `connection class corrupted` in arangosh
when just starting the arangosh without a connection to a server and running
code such as `require("internal")`, the shell always printed "connection class
corrupted", which was somewhat misleading.
* add separate option `--query.slow-streaming-threshold` for tracking slow
streaming queries with a different timeout value
* increase maximum number of collections/shards in an AQL query from 256 to 2048
* don't rely on `_modules` collection being present and usable for arangod startup
* force connection timeout to be 7 seconds to allow libcurl time to retry lost DNS
queries.
* fixes a routing issue within the web ui after the use of views
* fixes some graph data parsing issues in the ui, e.g. cleaning up duplicate
edges inside the graph viewer.
* in a cluster environment, the arangod process now exits if wrong credentials
are used during the startup process.
* added option `--rocksdb.total-write-buffer-size` to limit total memory usage
across all RocksDB in-memory write buffers
* suppress warnings from statistics background threads such as
`WARNING caught exception during statistics processing: Expecting Object`
during version upgrade
v3.4.0-rc.3 (2018-10-23)
------------------------
* fixed handling of broken Foxx services
Installation now also fails when the service encounters an error when
executed. Upgrading or replacing with a broken service will still result
in the broken services being installed.
* restored error pages for broken Foxx services
Services that could not be executed will now show an error page (with helpful
information if development mode is enabled) instead of a generic 404 response.
Requests to the service that do not prefer HTML (i.e. not a browser window)
will receive a JSON formatted 503 error response instead.
* added support for `force` flag when upgrading Foxx services
Using the `force` flag when upgrading or replacing a service falls back to
installing the service if it does not already exist.
* The order of JSON object attribute keys in JSON return values will now be
"random" in more cases. In JSON, there is no defined order for object attribute
keys anyway, so ArangoDB is taking the freedom to return the attribute keys in
a non-deterministic, seemingly unordered way.
* Fixed an AQL bug where the `optimize-traversals` rule was falsely applied to
extensions with inline expressions and thereby ignoring them
* fix side-effects of sorting larger arrays (>= 16 members) of constant literal
values in AQL, when the array was used not only for IN-value filtering but also
later in the query.
The array values were sorted so the IN-value lookup could use a binary search
instead of a linear search, but this did not take into account that the array
could have been used elsewhere in the query, e.g. as a return value. The fix
will create a copy of the array and sort the copy, leaving the original array
untouched.
* disallow empty LDAP password
* fixes validation of allowed or not allowed foxx service mount paths within
the Web UI
* The single database or single coordinator statistics in a cluster
environment within the Web UI sometimes got called way too often.
This caused artifacts in the graphs, which is now fixed.
* An aardvark statistics route could not collect and sum up the statistics of
all coordinators if one of them was ahead and had more results than the others
* Web UI now checks if server statistics are enabled before it sends its first
request to the statistics API
* fix internal issue #486: immediate deletion (right after creation) of
a view with a link to one collection and indexed data reports failure
but removes the link
* fix internal issue #480: link to a collection is not added to a view
if it was already added to other view
* fix internal issues #407, #445: limit ArangoSearch memory consumption
so that it won't cause OOM while indexing large collections
* upgraded arangodb starter version to 0.13.5
* removed undocumented `db.<view>.toArray()` function from ArangoShell
* prevent creation of collections and views with the same in cluster setups
* fixed issue #6770: document update: ignoreRevs parameter ignored
* added AQL query optimizer rules `simplify-conditions` and `fuse-filters`
* improve inter-server communication performance:
- move all response processing off Communicator's socket management thread
- create multiple Communicator objects with ClusterComm, route via round robin
- adjust Scheduler threads to always be active, and have designated priorities.
* fix internal issue #2770: the Query Profiling modal dialog in the Web UI
was slightly malformed.
* fix internal issue #2035: the Web UI now updates its indices view to check
whether new indices exist or not.
* fix internal issue #6808: newly created databases within the Web UI did not
appear when used Internet Explorer 11 as a browser.
* fix internal issue #2957: the Web UI was not able to display more than 1000
documents, even when it was set to a higher amount.
* fix internal issue #2688: the Web UI's graph viewer created malformed node
labels if a node was expanded multiple times.
* fix internal issue #2785: web ui's sort dialog sometimes got rendered, even
if it should not.
* fix internal issue #2764: the waitForSync property of a satellite collection
could not be changed via the Web UI
* dynamically manage libcurl's number of open connections to increase performance
by reducing the number of socket close and then reopen cycles
* recover short server id from agency after a restart of a cluster node
this fixes problems with short server ids being set to 0 after a node restart,
which then prevented cursor result load-forwarding between multiple coordinators
to work properly
this should fix arangojs#573
* increased default timeouts in replication
this decreases the chances of followers not getting in sync with leaders because
of replication operations timing out
* include forward-ported diagnostic options for debugging LDAP connections
* fixed internal issue #3065: fix variable replacements by the AQL query
optimizer in arangosearch view search conditions
The consequence of the missing replacements was that some queries using view
search conditions could have failed with error messages such as
"missing variable #3 (a) for node #7 (EnumerateViewNode) while planning registers"
* fixed internal issue #1983: the Web UI was showing a deletion confirmation
multiple times.
* Restricted usage of views in AQL, they will throw an error now
(e.g. "FOR v, e, p IN 1 OUTBOUND @start edgeCollection, view")
instead of failing the server.
* Allow VIEWs within the AQL "WITH" statement in cluster environment.
This will now prepare the query for all collections linked within a view.
(e.g. "WITH view FOR v, e, p IN OUTBOUND 'collectionInView/123' edgeCollection"
will now be executed properly and not fail with unregistered collection any more)
* Properly check permissions for all collections linked to a view when
instantiating an AQL query in cluster environment
* support installation of ArangoDB on Windows into directories with multibyte
character filenames on Windows platforms that used a non-UTF8-codepage
This was supported on other platforms before, but never worked for ArangoDB's
Windows version
* display shard synchronization progress for collections outside of the
`_system` database
* change memory protection settings for memory given back to by the bundled
JEMalloc memory allocator. This avoids splitting of existing memory mappings
due to changes of the protection settings
* added missing implementation for `DeleteRangeCF` in RocksDB WAL tailing handler
* fixed agents busy looping gossip
* handle missing `_frontend` collections gracefully
the `_frontend` system collection is not required for normal ArangoDB operations,
so if it is missing for whatever reason, ensure that normal operations can go
on.
v3.4.0-rc.2 (2018-09-30)
------------------------
* upgraded arangosync version to 0.6.0
* upgraded arangodb starter version to 0.13.3
* fixed issue #6611: Properly display JSON properties of user defined foxx services
configuration within the web UI
* improved shards display in web UI: included arrows to better visualize that
collection name sections can be expanded and collapsed
* added nesting support for `aql` template strings
* added support for `undefined` and AQL literals to `aql.literal`
* added `aql.join` function
* fixed issue #6583: Agency node segfaults if sent an authenticated HTTP
request is sent to its port
* fixed issue #6601: Context cancelled (never ending query)
* added more AQL query results cache inspection and control functionality
* fixed undefined behavior in AQL query result cache
* the query editor within the web UI is now catching HTTP 501 responses
properly
* added AQL VERSION function to return the server version as a string
* added startup parameter `--cluster.advertised-endpoints`
* AQL query optimizer now makes better choices regarding indexes to use in a
query when there are multiple competing indexes and some of them are prefixes
of others
In this case, the optimizer could have preferred indexes that covered less
attributes, but it should rather pick the indexes that covered more attributes.
For example, if there was an index on ["a"] and another index on ["a", "b"], then
previously the optimizer may have picked the index on just ["a"] instead the
index on ["a", "b"] for queries that used all index attributes but did range
queries on them (e.g. `FILTER doc.a == @val1 && doc.b >= @val2`).
* Added compression for the AQL intermediate results transfer in the cluster,
leading to less data being transferred between coordinator and database servers
in many cases
* forward-ported a bugfix from RocksDB (https://github.com/facebook/rocksdb/pull/4386)
that fixes range deletions (used internally in ArangoDB when dropping or truncating
collections)
The non-working range deletes could have triggered errors such as
`deletion check in index drop failed - not all documents in the index have been deleted.`
when dropping or truncating collections
* improve error messages in Windows installer
* allow retrying installation in Windows installer in case an existing database is still
running and needs to be manually shut down before continuing with the installation
* fix database backup functionality in Windows installer
* fixed memory leak in `/_api/batch` REST handler
* `db._profileQuery()` now also tracks operations triggered when using `LIMIT`
clauses in a query
* added proper error messages when using views as an argument to AQL functions
(doing so triggered an `internal error` before)
* fixed return value encoding for collection ids ("cid" attribute") in REST API
`/_api/replication/logger-follow`
* fixed dumping and restoring of views with arangodump and arangorestore
* fix replication from 3.3 to 3.4
* fixed some TLS errors that occurred when combining HTTPS/TLS transport with the
VelocyStream protocol (VST)
That combination could have led to spurious errors such as "TLS padding error"
or "Tag mismatch" and connections being closed
* make synchronous replication detect more error cases when followers cannot
apply the changes from the leader
* fixed issue #6379: RocksDB arangorestore time degeneration on dead documents
* fixed issue #6495: Document not found when removing records
* fixed undefined behavior in cluster plan-loading procedure that may have
unintentionally modified a shared structure
* reduce overhead of function initialization in AQL COLLECT aggregate functions,
for functions COUNT/LENGTH, SUM and AVG
this optimization will only be noticable when the COLLECT produces many groups
and the "hash" COLLECT variant is used
* fixed potential out-of-bounds access in admin log REST handler `/_admin/log`,
which could have led to the server returning an HTTP 500 error
* catch more exceptions in replication and handle them appropriately
* agency endpoint updates now go through RAFT
* fixed a cleanup issue in Current when a follower was removed from Plan
* catch exceptions in MaintenanceWorker thread
* fixed a bug in cleanOutServer which could lead to a cleaned out server
still being a follower for some shard
v3.4.0-rc.1 (2018-09-06)
------------------------
* Release Candidate for 3.4.0, please check the `ReleaseNotes/KnownIssues34.md`
file for a list of known issues.
* upgraded bundled RocksDB version to 5.16.0
* upgraded bundled Snappy compression library to 1.1.7
* fixed issue #5941: if using breadth first search in traversals uniqueness checks
on path (vertices and edges) have not been applied. In SmartGraphs the checks
have been executed properly.
* added more detailed progress output to arangorestore, showing the percentage of
how much data is restored for bigger collections plus a set of overview statistics
after each processed collection
* added option `--rocksdb.use-file-logging` to enable writing of RocksDB's own
informational LOG files into RocksDB's database directory.
This option is turned off by default, but can be enabled for debugging RocksDB
internals and performance.
* improved error messages when managing Foxx services
Install/replace/upgrade will now provide additional information when an error
is encountered during setup. Errors encountered during a `require` call will
also include information about the underlying cause in the error message.
* fixed some Foxx script names being displayed incorrectly in web UI and Foxx CLI
* major revision of the maintenance feature
* added `uuidv4` and `genRandomBytes` methods to crypto module
* added `hexSlice` methods `hexWrite` to JS Buffer type
* added `Buffer.from`, `Buffer.of`, `Buffer.alloc` and `Buffer.allocUnsafe`
for improved compatibility with Node.js
* Foxx HTTP API errors now log stacktraces
* fixed issue #5831: custom queries in the ui could not be loaded if the user
only has read access to the _system database.
* fixed issue #6128: ArangoDb Cluster: Task moved from DBS to Coordinator
* fixed some web ui action events related to Running Queries view and Slow
Queries History view
* fixed internal issue #2566: corrected web UI alignment of the nodes table
* fixed issue #5736: Foxx HTTP API responds with 500 error when request body
is too short
* fixed issue #6106: Arithmetic operator type casting documentation incorrect
* The arangosh now supports the velocystream transport protocol via the schemas
"vst+tcp://", "vst+ssl://", "vst+unix://" schemes.
* The server will no longer lowercase the input in --server.endpoint. This means
Unix domain socket paths will now be treated as specified, previously they were lowercased
* fixed logging of requests. A wrong log level was used
* fixed issue #5943: misplaced database ui icon and wrong cursor type were used
* fixed issue #5354: updated the web UI JSON editor, improved usability
* fixed issue #5648: fixed error message when saving unsupported document types
* fixed internal issue #2812: Cluster fails to create many indexes in parallel
* Added C++ implementation, load balancer support, and user restriction to Pregel API.
If an execution is accessed on a different coordinator than where it was
created, the request(s) will be forwarded to the correct coordinator. If an
execution is accessed by a different user than the one who created it, the
request will be denied.
* the AQL editor in the web UI now supports detailed AQL query profiling
* fixed issue #5884: Subquery nodes are no longer created on DBServers
* intermediate commits in the RocksDB engine are now only enabled in standalone AQL queries
(not within a JS transaction), standalone truncate as well as for the "import" API
* the AQL editor in the web UI now supports GeoJSON types and is able to render them.
* fixed issue #5035: fixed a vulnerability issue within the web ui's index view
* PR #5552: add "--latency true" option to arangoimport. Lists microsecond latency
* added `"pbkdf2"` method to `@arangodb/foxx/auth` module
* the `@arangodb/foxx/auth` module now uses a different method to generate salts,
so salts are no longer guaranteed to be alphanumeric
* fixed internal issue #2567: the Web UI was showing the possibility to move a shard
from a follower to the current leader
* Renamed RocksDB engine-specific statistics figure `rocksdb.block-cache-used`
to `rocksdb.block-cache-usage` in output of `db._engineStats()`
The new figure name is in line with the statistics that the RocksDB library
provides in its new versions.
* Added RocksDB engine-specific statistics figures `rocksdb.block-cache-capacity`,
`rocksdb.block-cache-pinned-usage` as well as level-specific figures
`rocksdb.num-files-at-level` and `rocksdb.compression-ratio-at-level` in
output of `db._engineStats()`
* Added RocksDB-engine configuration option `--rocksdb.block-align-data-blocks`
If set to true, data blocks are aligned on lesser of page size and block size,
which may waste some memory but may reduce the number of cross-page I/Os operations.
* Usage RocksDB format version 3 for new block-based tables
* Bugfix: The AQL syntax variants `UPDATE/REPLACE k WITH d` now correctly take
_rev from k instead of d (when ignoreRevs is false) and ignore d._rev.
* Added C++ implementation, load balancer support, and user restriction to tasks API
If a task is accessed on a different coordinator than where it was created,
the request(s) will be forwarded to the correct coordinator. If a
task is accessed by a different user than the one who created it, the request
will be denied.
* Added load balancer support and user-restriction to async jobs API.
If an async job is accessed on a different coordinator than where it was
created, the request(s) will be forwarded to the correct coordinator. If a
job is accessed by a different user than the one who created it, the request
will be denied.
* switch default storage engine from MMFiles to RocksDB
In ArangoDB 3.4, the default storage engine for new installations is the RocksDB
engine. This differs to previous versions (3.2 and 3.3), in which the default
storage engine was the MMFiles engine.
The MMFiles engine can still be explicitly selected as the storage engine for
all new installations. It's only that the "auto" setting for selecting the storage
engine will now use the RocksDB engine instead of MMFiles engine.
In the following scenarios, the effectively selected storage engine for new
installations will be RocksDB:
* `--server.storage-engine rocksdb`
* `--server.storage-engine auto`
* `--server.storage-engine` option not specified
The MMFiles storage engine will be selected for new installations only when
explicitly selected:
* `--server.storage-engine mmfiles`
On upgrade, any existing ArangoDB installation will keep its previously selected
storage engine. The change of the default storage engine is thus only relevant
for new ArangoDB installations and/or existing cluster setups for which new server
nodes get added later. All server nodes in a cluster setup should use the same
storage engine to work reliably. Using different storage engines in a cluster is
unsupported.
* added collection.indexes() as an alias for collection.getIndexes()
* disable V8 engine and JavaScript APIs for agency nodes
* renamed MMFiles engine compactor thread from "Compactor" to "MMFilesCompactor".
This change will be visible only on systems which allow assigning names to
threads.
* added configuration option `--rocksdb.sync-interval`
This option specifies interval (in milliseconds) that ArangoDB will use to
automatically synchronize data in RocksDB's write-ahead log (WAL) files to
disk. Automatic syncs will only be performed for not-yet synchronized data,
and only for operations that have been executed without the *waitForSync*
attribute.
Automatic synchronization is performed by a background thread. The default
sync interval is 100 milliseconds.
Note: this option is not supported on Windows platforms. Setting the sync
interval to a value greater 0 will produce a startup warning.
* added AQL functions `TO_BASE64`, `TO_HEX`, `ENCODE_URI_COMPONENT` and `SOUNDEX`
* PR #5857: RocksDB engine would frequently request a new DelayToken. This caused
excessive write delay on the next Put() call. Alternate approach taken.
* changed the thread handling in the scheduler. `--server.maximal-threads` will be
the maximum number of threads for the scheduler.
* The option `--server.threads` is now obsolete.
* use sparse indexes in more cases now, when it is clear that the index attribute
value cannot be null
* introduce SingleRemoteOperationNode via "optimize-cluster-single-document-operations"
optimizer rule, which triggers single document operations directly from the coordinator
instead of using a full-featured AQL setup. This saves cluster roundtrips.
Queries directly referencing the document key benefit from this:
UPDATE {_key: '1'} WITH {foo: 'bar'} IN collection RETURN OLD
* Added load balancer support and user-restriction to cursor API.
If a cursor is accessed on a different coordinator than where it was created,
the requests will be forwarded to the correct coordinator. If a cursor is
accessed by a different user than the one who created it, the request will
be denied.
* if authentication is turned on requests to databases by users with insufficient rights
will be answered with the HTTP forbidden (401) response.
* upgraded bundled RocksDB library version to 5.15
* added key generators `uuid` and `padded`
The `uuid` key generator generates universally unique 128 bit keys, which are
stored in hexadecimal human-readable format.
The `padded` key generator generates keys of a fixed length (16 bytes) in
ascending lexicographical sort order.
* The REST API of `/_admin/status` added: "operationMode" filed with same meaning as
the "mode" field and field "readOnly" that has the inverted meaning of the field
"writeOpsEnabled". The old field names will be deprecated in upcoming versions.
* added `COUNT_DISTINCT` AQL function
* make AQL optimizer rule `collect-in-cluster` optimize aggregation functions
`AVERAGE`, `VARIANCE`, `STDDEV`, `UNIQUE`, `SORTED_UNIQUE` and `COUNT_DISTINCT`
in a cluster by pushing parts of the aggregation onto the DB servers and only
doing the total aggregation on the coordinator
* replace JavaScript functions FULLTEXT, NEAR, WITHIN and WITHIN_RECTANGLE with
regular AQL subqueries via a new optimizer rule "replace-function-with-index".
* the existing "fulltext-index-optimizer" optimizer rule has been removed because its
duty is now handled by the "replace-function-with-index" rule.
* added option "--latency true" option to arangoimport. Lists microsecond latency
statistics on 10 second intervals.
* fixed internal issue #2256: ui, document id not showing up when deleting a document
* fixed internal issue #2163: wrong labels within foxx validation of service
input parameters
* fixed internal issue #2160: fixed misplaced tooltips in indices view
* Added exclusive option for rocksdb collections. Modifying AQL queries can
now set the exclusive option as well as it can be set on JavaScript transactions.
* added optimizer rule "optimize-subqueries", which makes qualifying subqueries
return less data
The rule fires in the following situations:
* in case only a few results are used from a non-modifying subquery, the rule
will add a LIMIT statement into the subquery. For example
LET docs = (
FOR doc IN collection
FILTER ...
RETURN doc
)
RETURN docs[0]
will be turned into
LET docs = (
FOR doc IN collection
FILTER ...
LIMIT 1
RETURN doc
)
RETURN docs[0]
Another optimization performed by this rule is to modify the result value
of subqueries in case only the number of results is checked later. For example
RETURN LENGTH(
FOR doc IN collection
FILTER ...
RETURN doc
)
will be turned into
RETURN LENGTH(
FOR doc IN collection
FILTER ...
RETURN true
)
This saves copying the document data from the subquery to the outer scope and may
enable follow-up optimizations.
* fixed Foxx queues bug when queues are created in a request handler with an
ArangoDB authentication header
* abort startup when using SSLv2 for a server endpoint, or when connecting with
a client tool via an SSLv2 connection.
SSLv2 has been disabled in the OpenSSL library by default in recent versions
because of security vulnerabilities inherent in this protocol.
As it is not safe at all to use this protocol, the support for it has also
been stopped in ArangoDB. End users that use SSLv2 for connecting to ArangoDB
should change the protocol from SSLv2 to TLSv12 if possible, by adjusting
the value of the `--ssl.protocol` startup option.
* added `overwrite` option to document insert operations to allow for easier syncing.
This implements almost the much inquired UPSERT. In reality it is a REPSERT
(replace/insert) because only replacement and not modification of documents
is possible. The option does not work in cluster collections with custom
sharding.
* added startup option `--log.escape`
This option toggles the escaping of log output.
If set to `true` (which is the default value), then the logging will work
as before, and the following characters in the log output are escaped:
* the carriage return character (hex 0d)
* the newline character (hex 0a)
* the tabstop character (hex 09)
* any other characters with an ordinal value less than hex 20
If the option is set to `false`, no characters are escaped. Characters with
an ordinal value less than hex 20 will not be printed in this mode but will
be replaced with a space character (hex 20).
A side effect of turning off the escaping is that it will reduce the CPU
overhead for the logging. However, this will only be noticable when logging
is set to a very verbose level (e.g. debug or trace).