From 331c0e5725e500d02169b143bba103fab7d9c64b Mon Sep 17 00:00:00 2001 From: Reneta Popova Date: Fri, 9 Feb 2024 16:37:25 +0200 Subject: [PATCH 1/2] Update the store format page --- .../database-internals/store-formats.adoc | 61 ++++++++++++++++++- 1 file changed, 58 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/database-internals/store-formats.adoc b/modules/ROOT/pages/database-internals/store-formats.adoc index 99088310f..4c8b0b234 100644 --- a/modules/ROOT/pages/database-internals/store-formats.adoc +++ b/modules/ROOT/pages/database-internals/store-formats.adoc @@ -11,6 +11,9 @@ The available formats are: Neo4j's newest store format is `block`. It has the best performance and supports the highest limits. +For example, compared to the existing record format, the `block` format has roughly 40% better performance when the whole graph is in memory. +That lead increases to 70% when only a third of the graph can fit in memory. + `block` format is intended to replace all the older formats. The `block` format uses a range of different data structures and inlining techniques to achieve data locality and store related data together on disk. This allows more related data to be fetched by fewer read operations, resulting in better resource utilization. @@ -25,14 +28,46 @@ The `high-limit` format allows more nodes and relationships to be stored than `s You can either set the store format when creating a new database or change the store format of an existing database. +The default store format of new databases is `aligned` and is controlled by the xref:configuration/configuration-settings.adoc#config_db.format[`db.format`] configuration setting in the _neo4j.conf_ file. + === Creating new databases -The default store format of new databases is controlled by the xref:configuration/configuration-settings.adoc#config_db.format[`db.format`] configuration setting in the _neo4j.conf_ file. -Or the store format can be passed as an argument to the tool creating the database, e.g. xref:tools/neo4j-admin/neo4j-admin-import.adoc#import-tool-full[`neo4j-admin database import`] or xref:backup-restore/copy-database.adoc[`neo4j-admin database copy`] commands. +There are several ways to create a new database in a specific store format: + +* Specify the store format when creating a new database using the `CREATE DATABASE` Cypher statement. +For example: ++ +[source,cypher] +---- +CREATE DATABASE blockdb OPTIONS {storeFormat: 'block'} +---- + +* Pass the store format as an argument when using the xref:tools/neo4j-admin/neo4j-admin-import.adoc#import-tool-full[`neo4j-admin database import`] command to create a new database. +For example: ++ +[source,shell] +---- +neo4j-admin database import ... --format=block blockdb +---- === Changing an existing database -Changing the store format is possible by re-writing all data in the new format using the xref:tools/neo4j-admin/migrate-database.adoc[`neo4j-admin database migrate`] or xref:backup-restore/copy-database.adoc[`neo4j-admin database copy`] commands. +Changing the store format is possible by re-writing all data in the new format using one of the following options: + +* Migrate an existing database using xref:tools/neo4j-admin/migrate-database.adoc[`neo4j-admin database migrate`] command. +For example: ++ +[source,shell] +---- +neo4j-admin database migrate --to-format="block" mydb +---- + +* Pass the new store format as an argument when using the xref:backup-restore/copy-database.adoc[`neo4j-admin database copy`] command to create a copy of an existing database. For example: ++ +[source,shell] +---- +neo4j-admin database copy --to-format="block" mydb blockdb +---- Changing the store format requires that: @@ -42,6 +77,26 @@ Changing the store format requires that: Changing store format can be a time-consuming operation, depending on the size and complexity of the data in the database. Performance heavily depends on the speed of the disk and the amount of available memory. +=== Verify the store format + +You can verify the store format of a database using the following Cypher: + +[source,cypher] +---- +SHOW DATABASES YIELD name, store +---- + +.Result +[role="queryresult"] +---- ++-----------------------------------+ +| name | store | ++-----------------------------------+ +| "blockdb" | "block-block-2b.1" | +| "neo4j" | "record-aligned-1.1" | +| "system" | "record-aligned-1.1" | ++-----------------------------------+ +---- === Effects of store format choice The store format is responsible for how data is written to disk and how to read it. From 4a62e18dd28f30a987bcbf8ae836d8cee4c05aaa Mon Sep 17 00:00:00 2001 From: Reneta Popova Date: Thu, 15 Feb 2024 12:16:09 +0000 Subject: [PATCH 2/2] apply suggestions from PR review and further improvements --- .../database-internals/store-formats.adoc | 57 ++++++++++--------- .../tools/neo4j-admin/neo4j-admin-import.adoc | 2 +- 2 files changed, 32 insertions(+), 27 deletions(-) diff --git a/modules/ROOT/pages/database-internals/store-formats.adoc b/modules/ROOT/pages/database-internals/store-formats.adoc index 4c8b0b234..77fc4e001 100644 --- a/modules/ROOT/pages/database-internals/store-formats.adoc +++ b/modules/ROOT/pages/database-internals/store-formats.adoc @@ -11,11 +11,9 @@ The available formats are: Neo4j's newest store format is `block`. It has the best performance and supports the highest limits. -For example, compared to the existing record format, the `block` format has roughly 40% better performance when the whole graph is in memory. -That lead increases to 70% when only a third of the graph can fit in memory. -`block` format is intended to replace all the older formats. -The `block` format uses a range of different data structures and inlining techniques to achieve data locality and store related data together on disk. +The `block` format is intended to replace all the older formats. +It uses a range of different data structures and inlining techniques to achieve data locality and store related data together on disk. This allows more related data to be fetched by fewer read operations, resulting in better resource utilization. The older store formats are based on the original `standard` format, each with different features and limitations. @@ -28,29 +26,31 @@ The `high-limit` format allows more nodes and relationships to be stored than `s You can either set the store format when creating a new database or change the store format of an existing database. -The default store format of new databases is `aligned` and is controlled by the xref:configuration/configuration-settings.adoc#config_db.format[`db.format`] configuration setting in the _neo4j.conf_ file. - === Creating new databases -There are several ways to create a new database in a specific store format: +The default store format for all new databases is `aligned`. +If you want to change it, you can set a new value for the xref:configuration/configuration-settings.adoc#config_db.format[`db.format`] configuration in the _neo4j.conf_ file. -* Specify the store format when creating a new database using the `CREATE DATABASE` Cypher statement. -For example: -+ -[source,cypher] ----- -CREATE DATABASE blockdb OPTIONS {storeFormat: 'block'} ----- +You can also create a new database on a specific store format by passing the new format as an argument to the command creating the database, for example, xref:tools/neo4j-admin/neo4j-admin-import.adoc#import-tool-full[`neo4j-admin database import full`] or xref:backup-restore/copy-database.adoc[`neo4j-admin database copy`] commands. -* Pass the store format as an argument when using the xref:tools/neo4j-admin/neo4j-admin-import.adoc#import-tool-full[`neo4j-admin database import`] command to create a new database. For example: -+ + [source,shell] ---- -neo4j-admin database import ... --format=block blockdb +neo4j-admin database import full ... --format=block blockdb ---- -=== Changing an existing database +//There are several ways to create a new database in a specific store format: + +// * Specify the store format when creating a new database using the `CREATE DATABASE` Cypher statement. +// For example: +// + +// [source,cypher] +// ---- +// CREATE DATABASE blockdb OPTIONS {storeFormat: 'block'} +// ---- + +=== Changing the format of existing databases Changing the store format is possible by re-writing all data in the new format using one of the following options: @@ -62,7 +62,8 @@ For example: neo4j-admin database migrate --to-format="block" mydb ---- -* Pass the new store format as an argument when using the xref:backup-restore/copy-database.adoc[`neo4j-admin database copy`] command to create a copy of an existing database. For example: +* Pass the new store format as an argument when using the xref:backup-restore/copy-database.adoc[`neo4j-admin database copy`] command to create a copy of an existing database. +For example: + [source,shell] ---- @@ -89,14 +90,18 @@ SHOW DATABASES YIELD name, store .Result [role="queryresult"] ---- -+-----------------------------------+ -| name | store | -+-----------------------------------+ -| "blockdb" | "block-block-2b.1" | -| "neo4j" | "record-aligned-1.1" | -| "system" | "record-aligned-1.1" | -+-----------------------------------+ ++----------------------------------+ +| name | store | ++----------------------------------+ +| "blockdb" | "block-block-1.1" | +| "neo4j" | "record-aligned-1.1" | +| "system" | "record-aligned-1.1" | ++----------------------------------+ ---- + +Additionally, you can use the `neo4j-admin database info` command to get detailed information about the store format of a database. +For details, see xref:tools/neo4j-admin/neo4j-admin-store-info.adoc[Display store information]. + === Effects of store format choice The store format is responsible for how data is written to disk and how to read it. diff --git a/modules/ROOT/pages/tools/neo4j-admin/neo4j-admin-import.adoc b/modules/ROOT/pages/tools/neo4j-admin/neo4j-admin-import.adoc index 54a759793..dff51c04c 100644 --- a/modules/ROOT/pages/tools/neo4j-admin/neo4j-admin-import.adoc +++ b/modules/ROOT/pages/tools/neo4j-admin/neo4j-admin-import.adoc @@ -409,7 +409,7 @@ label:enterprise-only[] [WARNING] ==== -Incremental import is not supported for `block` format yet. +Incremental import does not support `block` format. ==== Incremental import allows you to incorporate large amounts of data in batches into the graph.