Skip to content

Neo4j 3.0 changelog

Neo Technology Build Agent edited this page Oct 3, 2017 · 8 revisions

3.0.12

Kernel

  • Close statement in node relationships iterable's #9752

Cypher

  • TransactionBoundQueryContext no longer creates new statements #9790

Bolt Server

  • upgrade jetty server to 9.2 #10114

3.0.11

Kernel

  • Open legacy index in writable mode for non single instance read only environments #9688

3.0.10

Kernel

  • Allow skip bad entries logging during import #9103
  • Resolves a deadlock scenario applying constraint #9110
  • Don't cache unchanged records in DirectRecordAccess #9386
  • Fix out-of-bounds issue when applying fixed-reference high-limit format records during recovery or cluster store-copy. Bumps transaction log version. #9427

Cypher

  • Fixed an unnecessary error when loading an empty file using LOAD CSV WITH HEADERS #9183
  • Fixed error with accessing maps and lists with null, so it now evaluates to null instead #9272

Security

  • Add security headers when returning static assets #8816

Tools

  • neo4j-import: Ability to configure import input data buffer size using --read-buffer-size #9107

Docker

  • Configuration can be set from environment variables #83
  • Fixed issue which prevented dbms.mode=ARBITER from working #89
  • Fixed setting config values via environment values #91

Misc

  • License Lucene Index Upgrade as GPL #9529

3.0.9

Kernel

  • Fix a bug that on rare occasions could lead to leaking paged memory. #8499
  • Fix an issue in batch importer where id duplication detection would miss some 1 in 10.000 ids #8731
  • Makes CSV backslash-escaped quotes parsing optional #8789
  • Improve legacy index restoreSortFields speed. Fix for GH issue #8840 #8887

Cypher

  • Better error messages for property assignment to invalid collections #8513
  • Attempting to apply STARTS WITH, ENDS WITH, or CONTAINS on a non-string now returns null #8517
  • Fix eagerness for labels() function #8575
  • Remove cypher dependency on kernel IndexDescriptor #8616
  • Fixed cost planner ensuring concurrency guarantees on MERGE relationship #8629
  • Fixed bug with relationship direction in create unique with path variable #8678
  • Upgrade Neo4j 3.0 to support Cypher compiler for Neo4j 2.3.9 #8690
  • Fixed shortestPath planning when preceded by unwind #8699
  • Throw exception when calling shortestPath with same start and end node #8751
  • Fix bug where profile was reporting DB hits on the wrong pipe #8892
  • Added a new config setting dbms.import.csv.legacy_quote_escaping to select whether to conform to the standard https://tools.ietf.org/html/rfc4180 for interpreting escaped quotation characters in CSV files loaded using LOAD CSV. Setting this to false will use the standard, interpreting repeated quotes '""' as a single in-lined quote, while true will use the legacy convention originally supported in Neo4j 3.0 and 3.1, allowing a backslash to include quotes in-lined in fields. #8903
  • Cypher now reports the correct count for self-relationships (loops) #8930
  • Cypher now supports an updated parameter syntax $param and it is no longer possible to start symbolic names (variables, property keys, parameter names) with a currency symbol #8931

Packaging

  • Fixed log rotation on Windows #8871
  • Fixed location of logs on Windows #8884
  • Fixed directories specified in Debian/RPM config #8895

Security

  • Generated SSL certificate files should only be readable by owner. #8510

High Availability

  • The master of a two instance cluster will now serve writes on slave failure #8986
  • Cluster formation decision depends on initial_hosts contents #9007

Tools

  • Fixed handling of paths with spaces in them on Windows #8514
  • Ensured queries executed through neo4j shell are logged #8735
  • neo4j start once again prints correct address on startup #8916
  • Fixed shutdown of neo4j-shell to unbind its port #8949

Misc

  • Improved hostname resolution for Graphite connections #8787
  • Fixes an issue where adding multiple key/value pairs for a node or relationship for a legacy index in separate transactions would duplicate some internal keys by mistake so that the lucene documents wouldn't be seen as empty when it really was empty later on. This would make such documents, which should have been removed, lingering in the index. #8862
  • Fixes an issue where removing one key from a node or relationship which had multiple keys auto-indexed would remove all other keys for that entity from the auto-index too by mistake. #8863

3.0.8

Kernel

  • Fixes importer issue which could hang shutdown during panic #8198
  • Setting dbms.tracer=null will no longer prevent the database from starting when metrics are enabled #8402
  • Fix a bug that could prevent recovery from finding the latest check point record in the logs, preventing adequate recovery of the store. #8418
  • Update legacy indexes to preserve documents sort fields on update #8462
  • Add deferred locks to a list of enterprise edition dependencies #8465

Cypher

  • Fix a bug where aliasing of aggregations caused queries to fail in semantic checking. #8278

  • Fixes a bug with eagerness on property reads. #8295

    Previous to this commit, the eagerness analysis system has been working with the assumption that the QueryGraph encodes all reads being done in a PlannerQuery. This is not correct, the QueryHorizon may also encode reads. This commit fixes eagerness issues related to property reads in the horizon, when those conflict with writes being done in the QueryGraph.

    Still left to do are reads made by functions in the horizon (labels(), properties()). Additionally, potential conflicts between reads in horizon and subsequent writes in the tail(s) of the PQs.

    Also fixed for the rule planner.

  • Fixes a bug whereby previously-bound nodes were created when using FOREACH on a mixed-type collection #8367

    Foreach on a mixed/Any type collection containing nodes should not create nodes already bound by the loop variable.

    Switches to use SemanticTable.isNode(_:String) which correctly infers node type.

  • Fixed error in relationship direction in VarLengthExpand description #8368

  • RETURN * with ORDER BY should not leak internal variable names in the final result due to AST rewriting. #8389

  • Ensures that shortest path enforces relationship uniqueness when using the exhaustive fallback plan. This only applies to cases where a predicate require the exhaustive fallback plan. #8395

  • Make sure all expressions are shown in plan descriptions #8421

  • Upgrade Cypher compiler dependency to 2.3.8 #8487

Packaging

  • Print a helpful message if JAVA_HOME is incorrectly defined #8445

Bolt Server

  • Give client error message on all failures #8196

3.0.7

Kernel

  • Add possibility to use label store as a source of affected nodes for schema index creation. #7915
  • Lock node without relationships during detach delete. Fixes: #7771 #7991
  • Retry count store lookup in case of concurrent state change. #8003
  • Improved message when store upgrade can't be done #8023
  • Fix problem where terminating transaction from a different thread could end up terminating wrong transaction when using nested transactions. This could occur when one user attempts to terminate another user's transaction, but incorrectly ends up having their own transaction terminated. #8084
  • Fix number of fields growth in legacy indexes #8101

Cypher

  • Tweaks to query re-planning #7881

  • Fixes #7903 - resolves a type-checking bug for toInt() #7982

  • Refactor type checking for functions and expressions #7989

  • Fix type checking bugs related to mixed-type literal lists #7992

  • Make IN work with list slices and subscripts #8015

  • Faster planning for cartesian products #8046

  • Fix bug around IN predicate in Cypher #8053

  • Ensures property tokens are not attempted to be created on read operations #8100

  • asPrivateType not handling lists and maps #8225

  • Fixes a bug whereby the plan description for NestedExpression would fail #8241

    Missing implementation of toString in NestedPipeExpression leads to regex-failure when trying to remove generated variables from the plan description and ending up with a java.lang.IllegalArgumentException: Illegal group reference instead of showing the plan.

  • Fixes #8239 - resolves an issue when attempting to access the labels of a node that has just been deleted #8256

    If the node is deleted at the same time as we are trying to access the labels of that node there is a significant risk that we will have a NullPointerException. Instead of an error we should just return an empty iterator for that case.

High Availability

  • Includes reason for master replying NOT_LOCKED #7975
  • Increases the default ha.heartbeat_timeout value to 40s #8081
  • Adds configuration settings for ordering branch handling and store copying on slaves #8234

Tools

  • Fixed a race condition in neoj4-import with ID groups which would have some correct relationships considered invalid and logged into bad.log instead #8135

Config

  • Fixed parsing of config values with trailing spaces #8149 #8165

Browser

  • Cypher frame now only loads ascii table results when required #242
  • Update neo4j-driver to 1.0.4 #244

Bolt Server

  • Race conditions in ChunkedOutput #7963
  • Nicer error message when using null keys in maps #8169

Docker

  • Listen on localhost only when setting credentials #53

3.0.6

Kernel

  • Fixes #7204 an issue where log rotation sometimes would cut a transaction in half making it problematic for recovery to read such transactions. Transactions are now never split into multiple log files. Any db experiencing this issue can now also start that db with this fix included due to improved log reading abilities. #7895
  • (Enterprise-only) Bumps store version for change in high-limit format. This requires setting dbms.allow_format_migration=true config for stores using that format, although no actual store upgrade will be performed since it's a backwards compatible change. The store will be marked as being of the newer version so that older version can't start on an upgraded store. #7916

Cypher

  • Handle equivalence of collections containing null #7936

Packaging

  • Debian packages now provide upgrade instructions when upgrading from 2.x to 3.x #7695
  • Fixed a bug in the Debian packages which caused an "upgrading from 2.x" warning to be printed even when upgrading from 3.x #7918

High Availability

  • Update ha component lifecycle to make sure we have only one active update puller on slaves #7909

3.0.5

Kernel

  • Don't take locks for entities created in the same transaction. #7628
  • allow to access transaction id and transaction commit time on afterCommit in TransactionEventHandlers #7632
  • Open lucene indexes in read only mode when database started in read only mode (not applicable to ha setup) to prevent cases when usage of interrupts will result in deleted index. #7651
  • Open lucene indexes in read only mode when database started in read only mode (not applicable to ha setup) to prevent cases when usage of interrupts will result in deleted index. #7657
  • Speeding up enterprise 'high limit' record format by adding possibility to store small records in a fixed references form to avoid variable length encoding/decoding costs #7670
  • Enhance execution guard to be applicable for all transactions. Introduce #dbms.transaction.timeout system property to configure transaction timeout. #7739
  • Automatically rebuild indexes that fail to open on startup. #7745
  • Faster import tool due to ability to better utilize all available resources and higher I/O saturation. #7790
  • Fixes an issue where multi-partition indexes would, during startup, sometimes not be recognized as ONLINE when in fact they were. This would result in an unnecessary full population of the index #7826

Cypher

  • Fixed inability to use returned point types as parameters to later queries #7757

Server

  • Fixed misleading error message on https port conflict #7854

Misc

  • Fix missing config-migrator in neo4j-desktop #7517
  • Improves the performance of CSV input data reading by parsing data in parallel if multi-line fields are disallowed. #7605

3.0.4

Kernel

  • Transaction termination aware locks #7343
  • Fixes issue with relationship group type overflow #7363
  • Safe freeing of ids for HA slaves #7477
  • Safe freeing of ids for HA slaves #7511
  • Allow relationship ids to be reused #7555
  • Fixed exception thrown when iterating over reverseNodes() if Path was a single node long #7609 #7619
  • Fixes an issue where failure during forcing of transaction log after multiple concurrent threads appending to it wouldn't have all threads notice the kernel panic before marking their transactions as committed. In rare cases the effect may be that there could be a skipped transaction id in the otherwise sequential gap-free transaction log. #7621

Cypher

  • Improve the performance of IN checking. #7410

Packaging

  • Allow Debian package users to configure shutdown parameters via defaults #7323

Tools

  • Fixed an argument parsing bug in neo4j-import when multiple csv-files were specified on Windows #7292
  • Relationship groups chains are defragmented so that dense nodes are read more efficiently. This was recently changed as part of importing relationships per type due to high memory usage. With this change the memory requirements are unchanged, but adds the benefit of relationship groups being co-located on disk per node. #7435

Browser

  • Update Neo4j JS Driver to 1.0.2 #179
  • Update neo4j js driver to 1.0.3 #189

Misc

  • Fixes a problem where deleted property record ids may be reused twice after a role switch in HA, causing properties to be overwritten #7342
  • Made possible to allow node id reuse using 'dbms.ids.reuse.types.override' setting. #7600

3.0.3

Kernel

  • Automatic detection of record format #7141
  • Update default settings for non unique index sampling, sampler set initial size evaluation. #7160
  • Fix very rare decoding bug with dynamic records, where unused records would temporarily appear to be in use and contain garbage data #7310
  • Reintroduced read locks on property reads until discovered issues have been resolved #7348

Cypher

  • performance fix for toInt in Cypher #7247
  • Add special faster handling of IN when checking constant collections #7253
  • Apply predicates in VarExpand, not in filter #7294

Packaging

  • Fix: NEO4J_ULIMIT_NOFILE is now respected on Debian package service restart #7269

Tools

  • Errors related to input data will not print stack trace unless supplying --stacktrace #7263

Misc

  • Improved logging in online backup #7287

3.0.2

Kernel

  • Separate checkpoint triggers for backup and store copy #6981

  • Improve error message for incorrect store format #7007

  • Imports relationships per type #7026

  • Fix NPE after Recovery #7049

  • A little faster decoding of short string/array #7054

  • Handling of unused records in property chain #7060

  • Safe freeing of aggressively reused ids #7072

  • Fix rare record decoding exceptions caused by page reads racing with page writes to the same record, or page eviction. #7089

  • Prevents null from being added to legacy indexes #7097

  • Fix problem running MERGE inside FOREACH in compatibility mode #7146

  • Fix bug with partitioned index document update when document will be added to each partition. #7153

  • Reading properties scales better by no longer needing to acquire read locks. #7179

    This also removes experimental.use_read_locks_on_property_reads setting.

  • Fixes issue with map size in IndexDefineCommand #7213

  • Fixes an issue with wrong degrees in import-tool #7224

Cypher

  • Fix IDP solver configs #7047
  • Do not return internal point type #7104
  • Fix #7046: Cannot return paths from procedures #7105
  • Remove compiled runtime from 2.3 #7107
  • Fix problem with PERIODIC COMMIT in the shell #7109
  • Fix #7087: Procedures yielding collection of nodes are treated as sin… #7112
  • Improved error handling for spatial points #7120
  • Better error messages for percentile functions #7142
  • Misc improvements for LOAD CSV #7149
  • Fix issue when removing properties from variables that have unknown type #7152
  • NodeIndexScan should not claim to solve regexp #7157
  • Resolved ambiguity with Geographic and Cartesian CRS #7165
  • Allow ToString to defer some type checks to runtime #7169
  • Fix bug in the IN operator #7190
  • Error message for aggregation as procedure input #7197
  • Set a user-agent in http when using load csv #7210
  • Handle deleted entities in property operations #7217
  • Allow auto-parametrization to handle lists #7248

Packaging

  • Added ability to set max number of open files in /etc/default/neo4j (default behavior remains unchanged) #7095
  • Neo4j can now be started on OS X if only Java 8 is installed #6895 #7113
  • Work around for OpenJDK9 being in Ubuntu 16.04 repos #7188 #7222
  • Fix path resolution in service scripts #7231

High Availability

  • Introduce new parameter max_survivable_failed_members. #7011

Tools

  • Consistency checker defaults logs directory to db directory #6935
  • Do not resolve filepath '-' in neo4j-shell #7090

Config

  • Rename the old dbms.querylog.filename setting to mark it as internal #7093

Misc

  • Added print of Neo4j version #7057
  • use GeoJSON for point serialisation in rest #7119

3.0.1

Cypher

  • Deleting a node/relationship should never fail #6928
  • Add missing initialization of variables on each iteration #7061

3.0.0

Kernel

  • Dynamic configuration of record format specific settings #6864
  • Restrict custom IO configurations from performing store copy. #6901
  • Fix deadlock caused by different lock ordering on nodes for relationships #6930
  • Add a default configuration for dbms.checkpoint.iops.limit in EE #6949
  • Switch to standard record format in all editions and tools #6953
  • Safe percentage reporting of migration step, fix progress monitor for batch import #6971
  • Increase default backup timeout #6988

Cypher

  • Fix computing effects in DeleteEntityAction #6896
  • Support for CALL in shell #6916
  • Fix the logic around isolating aggregations #6917
  • Fix bug in rule planner pattern matcher #6922
  • Optimize Cypher aggregation memory usage #6948
  • Fix Namespacer handling for ProcedureResultItems in double CALL #6950
  • Fix sampling of lucene index size #6963
  • Fix a transaction leak in the procedure result #6973
  • Fixes around PERIODIC COMMIT and transactions #6974
  • Fix periodic commit in server #6995
  • Revert change in NiceHasher causing regression #6996
  • Fix transaction handling for PERIODIC COMMIT in Bolt server #6997

Browser

  • Update JS driver to 1.0.0 #105

Bolt Server

  • Packer should not fail on deleted entities #6960

3.0.0-RC1

Kernel

  • Bigger page size during import #6611
  • Process all property updates from particular node as single update during index population #6624
  • Fixes issue with incremental backup failing with invalid high id #6627
  • Fix record counting in legacy CC #6697
  • Adversarial and Devious PageCacheRule #6712
  • Fix for rare permanent self-deadlocks in the Enterprise lock manager #6654 #6754
  • Safe release of outdated entries in TimedRepository #6782
  • More efficient reference decoding #6819
  • Remove possibility to recover database directly from consistency checker, remove -recovery option. #6830
  • Change counts store to write -1 as trailer when empty. #6849

Cypher

  • Fix plan caching #6602
  • Improve selectivity estimation for inequalities #6605
  • Call procedures from within Cypher queries (systay remix) #6670
  • Use count store for all capitalizations of count(*) #6717
  • Make ExpandIntoPipe deterministic #6724
  • Procedures fail gracefully with a useful message when used incorrectly #6727
  • Fixes inconsistencies when compacting query plans #6732
  • When merging/creating a relationship do not create nodes if aliased in scope #6733
  • Using Kahan's algorithm to compute AvgFunction and SumFunction #6745
  • Fix label cardinality estimation on the rhs of apply when projecting #6751
  • Modulo function cannot only use doubles #6756
  • Improve performamnce for planning complex create/merge queries #6758
  • Optimize cost planner unnesting of eagerness #6801
  • Support set propety with generic expression representing nodes/rels #6802
  • Fix NPEs in NiceHasher and NiceHasherValue #6814
  • 3.0 sys.queryJmx #6835
  • Allow using CALL together with PeriodicCommit #6837
  • Enforce consistency when accessing array with non-integer index values #6858
  • Fix variable exposure through multiple projections #6867
  • Optimize query planning of "many merges" query #6870
  • Fix incorrect counts for relationships with direction BOTH #6881

Security

  • Reusing password should fail #6857
  • Renamed "sys." top-level procedure namespace to "dbms." #6880

Bolt Server

  • Send severity in notifications #6600
  • Include index state for db.indexes #6710
  • Include index state for db.indexes #6725
  • Include unique indexes in procedure #6736
  • sys.components should also return edition #6878

Misc

  • Fixed a bug in Windows with HA mode when connecting to self #6667
  • Fixes issue where some properties could get dropped during large imports #6692

3.0.0-M05

Kernel

  • Multi threaded index population #6334
  • Implements a record format with very high id limits #6353
  • Trust final non-static fields #6376
  • Make ENDS WITH an indexed backed predicate #6514
  • Fixes CC issue where some relationship inconsistencies may be overlooked #6517
  • Specify sender address for network communication #6562

Cypher

  • Replace 'collection' with 'list' in the manual #6415
  • Improve unwind to not materialize collections and avoid OOM #6463
  • More countstore shortcuts #6472
  • Fixed eagerness bug with deleted relationships #6549
  • Fixed eagerness on reads after deletes #6564
  • Cypher should be able to run queries when query cache size is 0 #6569
  • Fixed eagerness with deletes followed by reads #6571
  • Add properties function to Cypher #6581

3.0.0-M04

Kernel

  • Ability to specify db config in import tool #6284

Cypher

  • 2.3 idp long chains #6269
  • Add a configuration option to disable shortestpath fallback #6325
  • Relationship by id should work with arguments #6333
  • Make DISTINCT work with arrays inside nested structures #6340
  • 3.0 compact queryplan #6341
  • Stop warnings about literal maps keys #6347
  • Remove str() and cleanups #6352
  • Mark deleted entities as deleted in the string serialization #6354
  • Add support for periodic commit in Ronja #6356
  • Fix rule planner compatibility mode bug #6361
  • Do not create label ids when removing non-existant labels via Cypher #6367
  • Fix queries having multiple pattern sharing same rel names but different node names #6368
  • Do not create label ids when removing non-existant labels via Cypher #6369
  • Rename CTCollection to CTList #6388
  • On creation, no warning of non-existent of label, reltype, propkey ids #6422
  • Don't overflow on aggregation #6430

3.0.0-M03

Kernel

  • Make SlaveLockClient cope better with ComExceptions #6102
  • Fix reading of props with different block sizes #6141
  • 2.2 update pull and coms #6146
  • Ability to import > 2.4 billion nodes in import tool #6180
  • Skip strings and arrays that are longer then maximum allowed term size. #6213
  • Report error and starting line in import if quote is unbalanced #6217
  • Fix raw ASCII support in import tool #6221
  • Fix AStar result in case End equals Start #6223
  • Adaptive work-ahead queue size in ParallelBatchImporter #6225
  • Fix Integer overflow for large number of dense nodes #6253
  • Add a timer to arbiter's shutdown hook in case of deadlock #6293
  • Handle leading/trailing whitespace in importer #6321

Cypher

  • Support for MERGE (a)-[:T]->(b) in cost planner. #6069
  • Fix allShortestPath so it uses a fallback strategy #6121
  • Add index nested loop joins to Cypher #6162
  • Fix delete operation in cypher 2.0 compatibility mode #6206
  • Fix missing predicates from the solved query graph when using greedy #6210
  • Fix bug around disconnected patterns after WITH #6216
  • Make CONTAINS an index backed predicate in Cypher #6233
  • Improve error message for invalid unicode literal #6236
  • Fix issue with deleting paths #6237
  • Support LOAD CSV in cost-based planner #6242
  • Improve removeIdenticalPlans rewriter to avoid stack overflows #6257
  • Add test for MERGE inside UNWIND #6259
  • Add FOREACH to cost planner #6260

3.0.0-M02

Kernel

  • Log the first heartbeat after a series of missed #5629
  • Neo4j upgrade to lucene 5 #5861
  • Fix Cluster Uri regression again #5932
  • Fixed issue with last committed tx id in query planning #5935
  • Add an echo " done" and newline to stdout on stop #5949
  • Add check "hasTxStateWithChanges" when creating an unique constraint #5953
  • ALL and NONE predicates evaluated during shortest path traversal #5974
  • Deprecate untested code merged by mistake #5993
  • Enable the use of enterprise edition from neo4j-shell -path ... #5995
  • Fixed command aggregation for legacy indexes #6022
  • More robust placeholder parser in batch endpoint, fixes #5664 #6032
  • Accepts raw ascii code as delimiter in import tool #6034
  • Progress reporting on all migration participants #6050
  • Release lock client in KTI #6075
  • Add the option to trace ComExceptions #6117

Cypher

  • Correct status code when using hint with no index #5925
  • Fix plans generated with USING JOIN hints #5927
  • switch for running queries eagerly #5968
  • Lock incoming nodes from execution context #5999
  • Improve shortest paths in Cypher #6004
  • Value hash joins #6015
  • Remove the greedy planner #6051
  • Fix count query failures on empty database #6053
  • Fix bug with cost planner NodeIndexSeekByRange #6063
  • 3.0 Support REMOVE property in cost planner #6071
  • When logging cache evictions, use the query string instead of the AST #6100

Bolt Server

  • Added magic preamble to BOLT #6006
  • Unify BOLT into a single port #6020

Misc

  • Don't show a file limit warning on OS X when running neo4j console #6080

3.0.0-M01

Kernel

  • 3.0 count store for aggregations #5506
  • Introduce ascending-order fixed tx push strategy. #5712
  • Fix cluster uri resolution #5869
  • Add missing checks for tx changes before augmenting iterators #5888
  • Deprecate metrics reporting into a single csv file #5892

Cypher

  • 3.0 spatial point and distance #5520
  • Handle create () in cost planner #5534
  • Initial support create relationship in cost planner #5593
  • Support for DELETE in cost planner #5678
  • Support for SET label in cost planners #5700
  • Better error handling for percentileCont and percentileDisc #5701
  • Fixes AVG and SUM overflow for Cypher #5707
  • Add metrics about replan events #5717
  • Support REMOVE a:L in cost planner #5740
  • Show parameters in query logging #5747
  • Handle null in delete pipes #5810
  • Remove HAS #5816
  • Abort rewriting of predicates with too large DNF #5821
  • Support set properties in cost planner #5876
  • toString support for booleans #5879
  • Rename identifier to variable #5881
  • Prevent lost updates on concurrent set property #5891
  • Adjusted the toStringFunction exception to reflect it accepts Boolean #5900
  • support for MERGE () in cost planner #5907
  • Equality should treat chars as strings #5909