Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions modules/ROOT/content-nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,9 @@
** xref:manage-databases/errors.adoc[]
** xref:manage-databases/remote-alias.adoc[]
** xref:composite-databases/index.adoc[]
*** xref:composite-databases/introduction.adoc[]
*** xref:composite-databases/queries.adoc[]
*** xref:composite-databases/administration.adoc[]
*** xref:composite-databases/sharding-with-copy.adoc[]
*** xref:composite-databases/considerations.adoc[]
*** xref:composite-databases/queries.adoc[]

* xref:clustering/index.adoc[]
** xref:clustering/introduction.adoc[]
Expand Down
Binary file added modules/ROOT/images/federation-sharding.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 3 additions & 3 deletions modules/ROOT/pages/backup-restore/copy-database.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ However, the command will output Cypher statements, which you can run to recreat
====
* `neo4j-admin database copy` preserves the node IDs (unless `--compact-node-store` is used), but the relationships get new IDs.
* `neo4j-admin database copy` is not supported for use on the `system` database.
* `neo4j-admin database copy` is not supported for use on xref:composite-databases/introduction.adoc#composite-databases-concepts[composite databases].
It must be run directly on the databases that are part of a composite database.
* `neo4j-admin database copy` is not supported for use on xref:composite-databases/index.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
* `neo4j-admin database copy` is an IOPS-intensive process.
For more information, see <<copy-estimating-iops, Estimating the processing time>>.
====
Expand Down Expand Up @@ -60,7 +60,7 @@ The command will replace the original database with the newly created copy.

* `<fromDatabase>` -- Name of the source database.

* `<toDatabase>` -- Name of the target database.
* `<toDatabase>` -- Name of the target database.
If the same as `<fromDatabase>`, it is copied to a temporary location, by default the current working directory or the path as defined by `--temp-path`, before being moved to replace the original.

[[copy-database-command-options]]
Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/backup-restore/offline-backup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,6 @@ bin/neo4j-admin database dump neo4j --to-path=/dumps/neo4j

[NOTE]
====
`neo4j-admin database dump` cannot be applied to xref:composite-databases/introduction.adoc#composite-databases-concepts[composite databases].
It must be run directly on the databases that are part of a composite database.
`neo4j-admin database dump` cannot be applied to xref:composite-databases/index.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
====
6 changes: 3 additions & 3 deletions modules/ROOT/pages/backup-restore/planning.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ The recommended way to restore a database in a cluster is to xref:clustering/dat

[NOTE]
====
The Neo4j Admin commands `backup`, `restore`, `dump`, `load`, `copy`, and `check-consistency` are not supported for use on xref:composite-databases/introduction.adoc#composite-databases-concepts[composite databases].
They must be run directly on the databases that are part of a composite database.
The Neo4j Admin commands `backup`, `restore`, `dump`, `load`, `copy`, and `check-consistency` are not supported for use on xref:composite-databases/index.adoc[Composite databases].
They must be run directly on the databases that are associated with that Composite database.
====

.The following table describes the commands' capabilities and usage.
Expand Down Expand Up @@ -133,7 +133,7 @@ They must be run directly on the databases that are part of a composite database
| {check-mark}
| {cross-mark}

| Run against a composite databases
| Run against a Composite databases
| {cross-mark}
| {cross-mark}
| {cross-mark}
Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/backup-restore/restore-dump.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,6 @@ For more information, see xref:clustering/databases.adoc#cluster-seed[Seed a clu

[NOTE]
====
`neo4j-admin database load` cannot be applied to xref:composite-databases/introduction.adoc#composite-databases-concepts[composite databases].
It must be run directly on the databases that are part of a composite database.
`neo4j-admin database load` cannot be applied to xref:composite-databases/index.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
====
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ This node is either the leader for this database in a cluster _or_ this is a sta
| statusMessage | String | A message explaining the current state of the database, which could be an error encountered by the Neo4j server when transitioning the database to `requestedStatus`, if any.
| default | Boolean | Whether this database is the default for this DBMS.
| home | Boolean | Whether this database is the home database for this user.
| constituents | List<String> | A list of alias names making up this composite database, null for non-composite databases.
| constituents | List<String> | A list of alias names making up this Composite database, null for non-Composite databases.
|===

Note that for failed databases, `currentStatus` and `requestedStatus` are different.
Expand All @@ -46,7 +46,7 @@ For example:

The possible statuses are `initial`, `offline`, `store copying`, `deallocating`, `unknown`, `dirty`, and `quarantined`.

Additionally, note that databases hosted on servers that are offline are also returned by the `SHOW DATABASES` command.
Additionally, note that databases hosted on servers that are offline are also returned by the `SHOW DATABASES` command.
For such databases the `address` column displays `NULL`, the `currentStatus` column displays `unknown`, and the `statusMessage` displays `Server is unavailable`.

.Listing databases in standalone Neo4j
Expand Down Expand Up @@ -142,7 +142,7 @@ This node is either the leader for this database in a cluster _or_ this is a sta
| statusMessage | String | A message explaining the current state of the database, which could be an error encountered by the Neo4j server when transitioning the database to `requestedStatus`, if any.
| default | Boolean | Whether this database is the default for this DBMS.
| home | Boolean | Whether this database is the home database for this user.
| constituents | List<String> | A list of alias names making up this composite database, null for non-composite databases.
| constituents | List<String> | A list of alias names making up this Composite database, null for non-Composite databases.
|===

.Listing statuses for database _foo_
Expand Down Expand Up @@ -208,5 +208,5 @@ It is the same as creation time unless the database has been stopped at some poi
May be different between members when changes have not propagated. | 2342
| `replicationLag` | Integer | The difference in transaction numbers between this server and the writer of this database.
If this is persistently high, there may be a problem. | 1
| `constituents` | List<String> | A list of alias names making up this composite database, null for non-composite databases. | "[]"
| `constituents` | List<String> | A list of alias names making up this Composite database, null for non-Composite databases. | "[]"
|===
4 changes: 2 additions & 2 deletions modules/ROOT/pages/clustering/servers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ Server tags are used during database allocation and when configuring load balanc

[NOTE]
====
`allowedDatabases` and `deniedDatabases` do not affect composite databases, they are always available everywhere.
`allowedDatabases` and `deniedDatabases` do not affect Composite databases, they are always available everywhere.
====

If no options are set, a server can host any database in any mode.
Expand Down Expand Up @@ -255,7 +255,7 @@ For a description of all the server options (e.g., server tags) that can be alte

[NOTE]
====
`allowedDatabases` and `deniedDatabases` do not affect composite databases, they are always available everywhere.
`allowedDatabases` and `deniedDatabases` do not affect Composite databases, they are always available everywhere.
====

As with the `DEALLOCATE DATABASES FROM SERVER ...` command, if the alteration of a server's options renders it impossible for the cluster to satisfy one or more of the databases' topologies, then the command fails and no changes are made.
Expand Down
93 changes: 93 additions & 0 deletions modules/ROOT/pages/composite-databases/administration.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
:description: This section describes the administration and operation of Composite databases.
[role=enterprise-edition]
[[composite-databases-administration]]
= Managing Composite databases

Composite databases are managed using administrative commands.
They are created with the link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/databases#administration-databases-create-composite-database[`CREATE COMPOSITE DATABASE`^] command.

== Create a Composite database

====
[source, cypher]
----
CREATE COMPOSITE DATABASE cineasts
----
====

Constituent graphs are added with the link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/aliases#alias-management-create-database-alias[`CREATE ALIAS`^] administrative command, for example:

== Create an alias on a Composite database

====
[source, cypher]
----
CREATE ALIAS cineasts.latest
FOR DATABASE movies2022
----
====

Aliases can also be created for databases on other DBMSs:

== Create an alias for a remote database on a Composite database

====
[source, cypher]
----
CREATE ALIAS cineasts.upcoming
FOR DATABASE upcoming
AT 'neo4j+s://other.dbms.com'
USER $user
PASSWORD $secretpassword
----
====

The link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/databases#administration-databases-show-databases[`SHOW DATABASE`^] administrative command includes Composite databases.

Their `type` is reported as `"composite"`, and the `constituents` column lists the names of the aliases contained.

== Show a Composite database

====
[source, cypher]
----
SHOW DATABASE cineasts YIELD name, type, constituents
----
----
+---------------------------------------------------------------------+
| name | type | constituents |
+---------------------------------------------------------------------+
| "cineasts" | "composite" | ["cineasts.latest", "cineasts.upcoming"] |
+---------------------------------------------------------------------+
----
====


The link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/aliases#alias-management-show-alias[`SHOW ALIASES FOR DATABASE`^] administrative command can be used to inspect aliases on Composite databases in further detail.

== Show Composite database aliases

====
[source, cypher]
----
SHOW ALIASES FOR DATABASE
----
----
+----------------------------------------------------------------------------------------+
| name | database | location | url | user |
+----------------------------------------------------------------------------------------+
| "cineasts.latest" | "movies2022" | "local" | NULL | NULL |
| "cineasts.upcoming" | "upcoming" | "remote" | "neo4j+s://other.dbms.com" | "cineast" |
+----------------------------------------------------------------------------------------+
----
====

For a full description of the administrative commands for managing Composite databases, see link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/databases[Cypher Manual -> Database management^].


[[composite-databases-connecting]]
== Connect to drivers and applications

Drivers and client applications connect to Composite databases just like standard databases.
For more information, see the respective link:{neo4j-docs-base-uri}/[Neo4j Driver manual^].
19 changes: 0 additions & 19 deletions modules/ROOT/pages/composite-databases/considerations.adoc

This file was deleted.

89 changes: 84 additions & 5 deletions modules/ROOT/pages/composite-databases/index.adoc
Original file line number Diff line number Diff line change
@@ -1,11 +1,90 @@
[role=enterprise-edition]
[[composite-databases]]
= Composite databases
:description: This section describes administration and operation of composite databases.
:description: An introduction to Composite databases.

* xref:composite-databases/introduction.adoc[Introduction]
* xref:composite-databases/queries.adoc[Queries]
* xref:composite-databases/considerations.adoc[Further considerations]
* xref:composite-databases/sharding-with-copy.adoc[Sharding data with the `copy` command]
[[composite-databases-introduction]]
== Introduction

A Composite database is a special type of database introduced in Neo4j 5.
It supersedes the previous Fabric implementation in Neo4j 4.

Fabric is the architectural design of a unified system that provides a single access point to local or distributed graph data.
It allows you to efficiently manage your data, computing resources, storage, and network traffic.

Composite databases are the means to access this partitioned data or graphs with a single Cypher query.

Composite databases *do not store data* independently.
They use _database aliases_, which can be local or remote.
For more information, see link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/aliases/[Database alias management^].

Composite databases are managed using Cypher administrative commands.
Therefore, you can create them as any other database in Neo4j standalone and cluster deployments without needing to deploy a dedicated proxy server (as with Fabric in Neo4j 4).
For a detailed example of how to create a Composite database and add database aliases to it, see xref:composite-databases/administration.adoc[Managing Composite databases].

The following table summarizes the similarities and differences between Composite databases in Neo4j 5 and Fabric in Neo4j 4:

.Composite database Neo4j 5 vs. Fabric Neo4j 4
[cols="<24s,38,38",frame="topbot",options="header"]
|===
| | Composite database (Neo4j 5) | Fabric (Neo4j 4)

| Data access
2+| Gives access to graphs found on other databases (local or remote).

| Data storage
2+| Does not store data independently.

| Deployment
| Both standalone and cluster deployments.
| Both standalone and cluster deployments. However, in a cluster deployment, the _fabric_ proxy must be deployed on a dedicated machine.

| Configuration
| Managed using Cypher commands. Composite databases are created with the `CREATE COMPOSITE DATABASE` command and database aliases are added with `CREATE ALIAS ..`.
| Managed through configuration settings. The Neo4j DBMSs that host the same _fabric_ database must have the same configuration settings. The configuration must be always kept in sync.

| Sharding an existing database
2+| With the help of the `neo4j-admin copy` command.

| Security credentials
| Composite databases use the same user credentials as the database aliases.
| The Neo4j DBMSs that host the same Fabric virtual database must have the same user credentials. Any change of password on a machine that is part of Fabric must be kept in sync and applied to all the Neo4j DBMSs that are part of Fabric.

| Database management
2+| Does not support database management. Any database management commands, index and constraint management commands, or user and security management commands must be issued directly to the DBMSs and databases, not the Composite databases.

| Transactions
2+| Only transactions with queries that read from multiple graphs, or read from multiple graphs and write to a single graph, are allowed.

| Neo4j embedded
2+| Not available when Neo4j is used as an embedded database in Java applications. It can be used only in a typical client/server mode when users connect to a Neo4j DBMS from their client application or tool via Bolt or HTTP protocol.
|===

== Concepts

The main concepts that are relevant to understand when working with Composite databases are:

Data Federation::
Data federation is when your data is in two *disjoint graphs* with *different labels and relationship types*.
For example, you have data about users with their location and data about the users' posts on different forums, and you want to query them together.

Data Sharding::
Data sharding is when you have two graphs that share the *same model* (same labels and relationship types) but contain *different data*.
For example, you can deploy shards on separate servers, splitting the load on resources and storage.
Or, you can deploy shards in different locations, to be able to manage them independently or split the load on network traffic.
An existing database can be sharded with the help of the `neo4j-admin database copy` command.
For an example, see xref:composite-databases/sharding-with-copy.adoc[Sharding data with the copy command].

Connecting data across graphs::
Because relationships cannot span across graphs, to query your data, you have to federate the graphs by
using a _proxy node_ modeling pattern, where nodes with a specific label must be present in both federated domains.
+
In one of the graphs, nodes with that specific label contain all the data related to that label, while in the other graph, the same label is associated with a proxy node that only contains the `<node>ID` property.
The `<node>ID` property allows you to link data across the graphs in this federation.

image::federation-sharding.png[title="Data federation and sharding", width=450, role=middle]

[TIP]
====
For a step-by-step tutorial on setting up and using a Composite database with federated and sharded data, see xref:tutorial/tutorial-composite-database.adoc[Set up and use a Composite database].
====
Loading