Neo4j 1.9 Enterprise changelog
HA:
- Index delete now propagates properly across all cluster nodes
- Slaves now retry switching, making possible overcoming temporary failures
- Remote transaction initialization is now explicitly marked and tracked, solving transaction leakage and reuse problems
- HA instances no longer require a snapshot on cluster join
- HA instances now properly switch to PENDING state when losing quorum
- Fixes save deadlock problem
- Transactions now happen in the context of a master epoch, to solve issues with transactions wrongly propagating between masters
- Introduces monitoring for remote calls between cluster instances
Kernel:
- Fixes issue with batch inserter which could lead to record not in use exception when adding properties
- Introduces monitoring for reads/writes to the logical log
Server:
- Fixes issue with transacions leaking when there is an exception while rendering the response
Backup:
- HA backup instances now are logged as backup instead of "-1"
Cluster:
- Fixes message reordering issue
- Fixes a Paxos instance reuse issue
- Cluster join denial now includes cluster configuration
- Fixes issues with proper Payload values passed around
HA:
- Removed all usage of getHostName() that might lead to problems on instances with improperly configured DNS
- The master instance will now always send out masterIsAvailable messages in response to masterIsElected, ensuring that instances in PENDING state will transition eventually to SLAVE
- Master instance now properly blocks transactions from starting after shutdown has started and waits on a timeout for running transactions to finish and be pushed to slaves before stopping completely
Kernel:
- Fixed NPE in PersistenceWindowPool when concurrently loading non-mapped windows
- Fixed potential deadlock in PersistenceWindowPool cause by negative mark count
- Performance optimizations for best-first selector
- Database now waits on a timeout for transactions to finish before actually shutting down, while preventing new transactions from starting up
Server:
- Shutdown hook is now properly removed on normal shutdown, removing potential thread leak which might prevent orderly exit of the VM process
Graph Algorithms:
- Fixed potential suboptimal path finding in A* executions for particular graph topologies. Also improved performance and memory consumption of A*
Cluster:
- Fixed a bug around channel handling which might lead to messages being delayed and leading to false heartbeat failures
- It is now possible to specify 0.0.0.0 for ha.cluster_server and properly wait for messages on all interfaces
- Removed all usage of getHostName() that might lead to problems on instances with improperly configured DNS
- Fixed election bug which might lead to infinite wait on elections
- Fixed election bug which might lead to votes being discarded and result in the wrong master being elected
- Instances no longer expect heartbeats from themselves so they can't timeout on their own
- Instances will now ignore suspicions from ex cluster members
- Instances will now ignore suspicions for themselves
Backup:
- Backup tool and command line utility now exit with status 1 in case the target store has mismatching store ID with the source.
Enterprise Server:
- Properly return UNKNOWN from the HA REST API when the instance is neither master or slave.
Cluster:
- Suspicions on failed instances are now printed only once, no longer spamming messages.log.
- Fixed issue where a failing cluster instance might delay messages between the rest of the instances.
Lucene index:
- Fixed issue where querying an index for ":" while having removed entities from it in the same transaction might result in exceptions.
Logging in console is now consistent across modules and more informative
HA:
- Fix for wrong snapshot processing on cluster join, leading to failure to switch to Slave
- Fix in SlaveIdGenerator which could lead to records being marked as not in use
- Important cluster events are now properly included in the console
Kernel:
- Remove hard dependency on Logback
- Introduction of logical_log_rotation_threshold to control log rotation.
- Autoconfigurator can handle cases where physical memory is less than the configured JVM heap
- Fixes index out of bounds errors when iterating over relationships
- Fixes race in persistent window pool which might lead to inconsistent data
Cypher:
- Fixes problems with extra columns showing up in some conditions
Server:
- Reintroduces, deprecated, the Configurator.getThirdpartyJaxRsClasses. Fixes, among other things, the authentication-extension
- Server now properly returns the version when asked through REST
Index:
- Stop keeping norms in Lucene indexes, saving space and memory
Com:
- Fix for a race between Slave shutting down a channel and Master reaping it, which led to exceptions on the Master side
Cluster:
- Fixes issue which might lead to eternal elections
- UUID for Available messages is now ignored, allowing for upgrades from 1.9.2
Cluster:
- Fixes bug where a single instance cluster will perpetually try to perform elections
Kernel:
- When flushing persistent windows, do so only if dirty. This is a performance improvement for certain scenarios.
- Fixes bug where loading relationship chains with high ids would cause them to be ignored when iterating over them.
Cypher:
- Fixes problem where Cypher would see changes while the query was running
Server:
- The Content-Type is now correctly used to set the text encoding, not Content-Encoding
- Fixes concurrency issue with generation of JSON responses
- HTTPS scheme is now properly propagated on batch requests
HA:
- Access guard is now raised during recovery
- Properly throttle push notifications from the master, preventing an exhaustion of resources and accompanying failures to pull from slaves
- Slaves will now not participate in elections if not eligible (e.g. when copying the store)
Kernel:
- Fixes bug where creating an Index with an invalid configuration would not properly clean up the configuration for it.
- Fixes race condition that occasionally would make Node.getSingleRelationship() mistakenly fail.
- Fixes compliance for non-logback logging implementations
- Cleanup of transactions that fail commit happens under the same lock as the commit
Cypher:
- Cypher execution results are now closeable
- Allow | to be used as a separator in foreach and extract+reduce
Server:
- Fixes bug where the last security rule would be the only one respected, if multiple would be present
- Support for chained certificates
- Enabled streaming support for paged traversals
Com:
- Client now uses heap buffers, reduces GC requirements and out of memory errors
- Master will now not timeout long running write operations, like store copying
- Redesigned ResourcePool, now it properly pools and reuses resources, should be more easy going on the garbage collector
Packaging:
- Improved handling of JAVA_HOME on OSX
Cluster:
- Coordinator doesn't ask itself for snapshots, solving a bug where starting an arbiter first would make the cluster unresponsive.
Enterprise server:
- Arbiter instances now log into data/log/arbiter/messages.log
Kernel:
- Performance improvement on initial loading of relationship types during startup
Backup:
- Reverts default backup port to 6362
Cluster:
- Upgrades to Netty 3.6.3
- Elections don't start when a quorum is not available, fixing a certain class of race conditions when quorum is lost
- The joining of an instance is now interpreted as a heartbeat, fixing a class of race conditions when an instance joins and immediately dies
Com:
- Upgrades to Netty 3.6.3
Kernel:
- Fixes race conditions on usage of persistence windows' byte buffers that occasionally led to buffer over/underflows.
Server:
- Create unique node with properties that have collection values properly casts them to arrays
- Data visualization editor now navigates back to the data browser on save and cancel
Cypher:
- Fixes problem when sending down custom queries to index providers
HA:
- Introduction of pseudo quorum writes. If half or more of the instances are unreachable, the instance will stop accepting write requests and all transactions will timeout.
Backup:
- -full and -incremental backup flags are deprecated, type of backup is automatically determined by the contents of the target directory
Cluster:
- Removes URL discovery mechanism for clustering
- Moves from URI as an instance id to integers - they must be specified manually. The new setting is ClusterSettings.server_id
- Instances can now be replaced if they have marked as fail if a new instance with the same id joins the cluster
- For a cluster to be formed now a majority of instances as configured in initial_hosts must be available
- Fixes around cluster formation from instances concurrently starting up
- Explicit join denial is now possible if an instance tries to join using a server_id already in use
Lucene Index:
- Lucene upgraded to 3.6.2
Server:
- Introduces new splash screen containing a guide to Neo4j. Several small aesthetic improvements
Cypher:
- Fixed #578 - problem with profiling queries that use LIMIT
- Fixes #550 - problem when using START after WITH
- Allows single node patterns in MATCH
- Fixes problem for some patterns and all-nodes start points
- Fixes #650 - issue when doing multiple aggregation expressions on the same identifier
- Added timestamp function
Packaging:
- plugins/ subdirectory is searched recursively for server plugins
Now compiles and runs under JDK 7 Introduces arbiter script for cluster client instances
HA:
- Slave only mode is now available
- HA server will now bind by default to all interfaces (0.0.0.0)
Backup:
- Backup server now binds by default to 127.0.0.1
Cluster:
- Election happens for all roles when an instance comes online
- Snapshots are no longer received automatically on cluster join
- Instances starting concurrently will now discover each other instead of each starting their own cluster
- Acceptor store now does not grow unbounded and is garbage collected
- Learn requests are made towards and responded from all cluster members
- Cluster server will now by default bind to all interfaces (0.0.0.0)
Com:
- Shutting down Server now takes proper care of executors and works with multiple versions of netty
Enterprise Server:
- HA info REST endpoints now are called /db/manage/server/ha/{ismaster,isslave}, with getMaster no longer existing.
Kernel:
- Concurrent modifications to relationship chains now do not lead to RecordNotInUse exceptions or cache poisoning
- Proper tx management will now make negative transaction counts impossible
- IndexProvider is now deprecated, replaced by KernelExtensionFactory
- Store locks are respected from read only instances too
- grab_file_lock configuration setting is now deprecated
Lucene Index:
- LuceneIndexProvider is now replaced by LuceneKernelExtension
Server:
- Added support for X-Forwarded-Host and X-Forwarded-Proto headers to allow parameterising of links in data for hosting behind proxy servers.
JMX:
- JMX will now provide info on all configuration values, including the defaults not explicitly set
Cypher:
- Fixes #450 - Predicates in WHERE are not evaluated correctly for certain patterns
- Fixes problem with null comparisons and optional relationships
- Made it possible to run the parser concurrently
- DISTINCT is now lazy, and keeps the incoming ordering
- Added execution plan descriptions to execution results
- Fixes #464 - getDeletedRelationships does not work for javacompat.ExecutionResult
- Fixes #535 - 'INTERNAL_SORT' columns show when doing RETURN * ORDER BY
- Added experimental profiled query support
- Fixes #489 - CREATE UNIQUE does not work as expected inside FOREACH
HA:
- Conflicting server IDs in a cluster will now make the latest joining instance to leave the cluster immediately and become non functional
Enterprise Server:
- Includes new service endpoint, typically found under /db/manage/server/ha/{isMaster,getMaster}, with HA related info.
Backup:
- online_backup_port is now depracated in favour of online_backup_server, works with a port range
- online backup is now enabled by default if possible
- Improved error reporting for backup tool
Cluster:
- Warning is now printed if an instance that is not part of ha.initial_hosts tries to join the cluster
- Snapshots are now prunned when received during cluster joining
- Logging cleanup
Kernel:
- Start entries are now explicitly marked as written, solves a bug that might cause recovery to fail
Server:
- Increased performance for rest-batch-operations by a factor of 100
Cypher:
- Clearer error message for when aggregations are used in ORDER BY and not in RETURN
- Fixes #394 - Identifiers inside of FOREACH get the wrong type
- Fixes #390 - IN/ANY/NONE/ANY/SINGLE causes RuntimeException for some patterns
- Fixes #387 - Some patterns produced "Unexpected traversal state" in Cypher
- Upgraded the Scala version to 2.10
- Fixes #355 - Collections in collections are still Scala collections instead of Java collections
Server:
- Pulled out Gremlin as separate plugin to support different versions
Cypher:
- Fixes #336 - Patterns that re-use a pattern node can produce non-existing matches
- Fixes #369 - The substring-function sometimes fails when no length is specified
HA:
- Lifecycle improvements, with proper removal of listeners on shutdown
- Cluster snapshot synchronization used for HA cluster membership
- Implements JMX changes
- IdGenerators instantiated properly in ModeSwitcher
- ZooKeeper compatibility layer introduced, allows for rolling upgrades from 1.8
- Ability to join a cluster even if there's an existing and empty database with a mismatching store ID. If the existing database isn't empty an error will be logged and database unable to start.
Cluster:
- Moved shared logic, commonly refered to as cluster client into a ClusterClient class.
- Reduced amount of threads used by Paxos by using fixed size thread pools
- Paxos cluster membership management imporoved by introducing memberIsUnavailable and cleanup of roles
Consistency Checker:
- Write out log file if inconsistencies are found.
- Improved configuration for memory mapping with MOST_FREQUENTLY_USED window pool.
- Consistency checker defaults to using the 'most frequently used' implementation when on Windows to avoid memory (un)mapping problems
Management:
- Adds serverId and current roles to the set of information available for cluster members
Kernel:
- Made sure that auto-indexing removed graph elements from indexes when they are delete
- OrderByTypeExpander keeps ordering of type AND direction
- Fixed an issue where a lock on a store was sometimes not released
- Old GraphDatabaseSetting classes are now wrappers for Settings instances
- Fixes an issue where an incomplete 2PC transaction could cause recovery to not be triggered
- Optimizations for read performance
- Cache refers to relationship types as ints instead of Strings.
- Binary search on sorted arrays for finding properties and relationship types in the cache.
- Less objects instantiated during getting and iterating relationships.
- Reduced number of map lookups significantly for a getProperty call, especially outside transaction or in transactions without any changes. (previously 8 lookups whereof 2 synchronized, now down to as low as 2)
- Uses ThreadLocal as mapper from thread to transaction
- Refactored LockReleaser into TransactionState, associated with each transaction instead
Server:
- Server now allows - under some specific circumstances - setting empty arrays as properties. Specifically, it is allowed if there is a pre-existing array property on the entity from which the server can infer the type of array to store.
- Traversal Javascript is now security sandboxed. It is possible to turn the sandbox off for the next two releases, for backwards compatibility.
Cypher:
- The traversal pattern matcher now supports variable length relationship patterns
- Fixes #946 - HAS(...) fails with ThisShouldNotHappenException for some patterns
- Made Cypher better at keeping a numeric type. Adding two integers now returns an integer, and not a double.
- Major refactoring to make certain Cypher is more lazy
- When asking for the top x rows by some value, Cypher will now only keep a list the size of x
- Fix so identifiers created inside a foreach can be used by other statements in the foreach
- Fix for problems when using patterns with multiple unnamed nodes
- Fixes problems when doing ORDER BY ... LIMIT x. When x is larger than the input set, sorting was never done.
Introduces consistency check and cluster components.
HA:
- Massive changes to stop using ZooKeeper as a coordination service and instead move to a custom Paxos implementation.
- Removed all ZooKeeper related functionality
- Added support classes for event processing from the cluster component
- All kernel parts with master/slave implementations are instantiated and managed separately and their switching happens based on cluster events
- HA management beans expose the new information properly
- Updates to documentation
Backup:
- Consistency check extracted into its own module (consistency-check).
- BackupTool extended to accept tuning parameters for consistency check.
Consistency check:
- Initial release (extracted from backup module).
- Added new consistency check implementation with better performance thanks to better usage of mapped memory.
- Consistency check can now accept tuning parameters to make best use of available resources.
- New WindowPool implementation for exclusive use underneath the consistency check.
Cluster:
- Initial release
Kernel:
- XaDatasources now implement lifecycle and their registration with XaDatasourceManager triggers TxManager recovery on startup
- Neo4j logical log now handles running out of disk space at a critical point gracefully.
- Kernel extensions are now factories that create instances which participate in the database's lifecycle
- Fixes a race condition around relationship chain loading from multiple threads on the same node
Cypher:
- Refactored the type system from the bottom up
- Predicates can now be returned and used to set properties
- Fixes #797: CREATE UNIQUE now makes sure used identifiers have the same properties even if they are re-used without the properties
- Added the traversal matcher, which uses the new traversal framework abilities to do pattern matching. It should provide for nice performance benefits for a lot of queries.
- Fixed #866: Changed the LRU cache to make it safe to use concurrently, and made the parser single threaded
- Added the reduce() functionality
- Made addition automatically convert numbers to strings when adding the two together
- Added the string functions: str(), replace(), substring(), left(), right(), ltrim(), rtrim(), trim(), lower(), upper()
- Added the possibility to use ORDER BY, SKIP and LIMIT together with WITH
- Fixes #904: CREATE UNIQUE doesn't work with parameter maps
- Fixes #908: Parameters do not survive WITH if it has aggregation
- SET can now be used to set properties on nodes and relationships from maps or other graph elements
Management:
- High Availability beans now expose information relevant to the new way clusters work
