Skip to content

Commit

Permalink
Extract kernel docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jjaderberg authored and benbc committed Sep 26, 2016
1 parent f6ab20c commit 4a87a77
Show file tree
Hide file tree
Showing 20 changed files with 168 additions and 81 deletions.
42 changes: 21 additions & 21 deletions manual/contents/pom.xml
Expand Up @@ -48,20 +48,6 @@
<classifier>docs</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>${neo4j.version}</version>
<classifier>docs</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>${neo4j.version}</version>
<classifier>test-sources</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-management</artifactId>
Expand Down Expand Up @@ -118,16 +104,23 @@
<classifier>docs</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-udc</artifactId>
<version>${neo4j.version}</version>
<classifier>docs</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j.doc</groupId>
<artifactId>neo4j-cypher-docs</artifactId>
<artifactId>neo4j-kernel-docs</artifactId>
<version>${neo4j.version}</version>
<classifier>sources</classifier>
<classifier>docs</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j.doc</groupId>
<artifactId>neo4j-cypher-docs</artifactId>
<artifactId>neo4j-kernel-docs</artifactId>
<version>${neo4j.version}</version>
<classifier>test-sources</classifier>
<scope>provided</scope>
Expand All @@ -136,12 +129,19 @@
<groupId>org.neo4j.doc</groupId>
<artifactId>neo4j-cypher-docs</artifactId>
<version>${neo4j.version}</version>
<classifier>docs</classifier>
<classifier>sources</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-udc</artifactId>
<groupId>org.neo4j.doc</groupId>
<artifactId>neo4j-cypher-docs</artifactId>
<version>${neo4j.version}</version>
<classifier>test-sources</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j.doc</groupId>
<artifactId>neo4j-cypher-docs</artifactId>
<version>${neo4j.version}</version>
<classifier>docs</classifier>
<scope>provided</scope>
Expand Down Expand Up @@ -349,7 +349,7 @@
<excludes>META-INF,META-INF/**,org/neo4j/kernel/impl/storemigration/legacystore/**</excludes>
<type>jar</type>
<outputDirectory>${docs.test-sources}</outputDirectory>
<includeArtifactIds>neo4j-cypher-docs,neo4j-examples,neo4j-harness-test,neo4j-server,neo4j-lucene-index,neo4j-backup,neo4j-kernel</includeArtifactIds>
<includeArtifactIds>neo4j-cypher-docs,neo4j-examples,neo4j-harness-test,neo4j-server,neo4j-lucene-index,neo4j-backup,neo4j-kernel-docs</includeArtifactIds>
</configuration>
</execution>
<execution>
Expand Down
2 changes: 1 addition & 1 deletion manual/contents/src/advanced/index.asciidoc
Expand Up @@ -31,7 +31,7 @@ include::{importdir}/neo4j-lucene-index-docs-jar/dev/index.asciidoc[]

:leveloffset: 1

include::{importdir}/neo4j-kernel-docs-jar/dev/batchinsert.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/dev/batchinsert.asciidoc[]

:leveloffset: 2

Expand Down
10 changes: 5 additions & 5 deletions manual/contents/src/operations/configuration.asciidoc
Expand Up @@ -9,21 +9,21 @@ The following sections describe how to tune these.

:leveloffset: 2

include::{importdir}/neo4j-kernel-docs-jar/ops/introduction.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/ops/introduction.asciidoc[]

:leveloffset: 2

include::{importdir}/neo4j-server-docs-jar/ops/server-performance.asciidoc[]

:leveloffset: 2

include::{importdir}/neo4j-kernel-docs-jar/ops/performance-guide.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/ops/performance-guide.asciidoc[]

include::{importdir}/neo4j-kernel-docs-jar/ops/logical-logs.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/ops/logical-logs.asciidoc[]

include::{importdir}/neo4j-kernel-docs-jar/ops/property-compression.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/ops/property-compression.asciidoc[]

include::{importdir}/neo4j-kernel-docs-jar/ops/io-examples.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/ops/io-examples.asciidoc[]

include::configuration-reference.asciidoc[]

Expand Down
2 changes: 1 addition & 1 deletion manual/contents/src/reference/index.asciidoc
Expand Up @@ -13,7 +13,7 @@ include::capabilities/cap-details.asciidoc[]

:leveloffset: 1

include::{importdir}/neo4j-kernel-docs-jar/dev/transactions.asciidoc[]
include::{importdir}/neo4j-kernel-docs-docs-jar/dev/transactions.asciidoc[]

:leveloffset: 1

Expand Down
4 changes: 4 additions & 0 deletions manual/kernel/LICENSES.txt
@@ -0,0 +1,4 @@
This file contains the full license text of the included third party
libraries. For an overview of the licenses see the NOTICE.txt file.


26 changes: 26 additions & 0 deletions manual/kernel/NOTICE.txt
@@ -0,0 +1,26 @@
Neo4j
Copyright © 2002-2016 Network Engine for Objects in Lund AB (referred to
in this notice as "Neo Technology")
[http://neotechnology.com]

This product includes software ("Software") developed by Neo Technology.

The software ("Software") is developed and owned by Network Engine
for Objects in Lund AB (referred to in this notice as "Neo Technology").
If you have executed an End User Software License and Services Agreement,
an OEM Software License and Support Services Agreement, or another
commercial license agreement (including an Evaluation Agreement) with
Neo Technology or one of its affiliates (each, a "Commercial Agreement"),
you may use the Software solely pursuant to the terms of the relevant
Commercial Agreement.

If you have not executed a Commercial Agreement with Neo Technology, the
Software is subject to the terms of the GNU AFFERO GENERAL PUBLIC LICENSE
Version 3 (http://www.fsf.org/licensing/licenses/agpl-3.0.html), included
in the LICENSE.txt file.

Full license texts are found in LICENSES.txt.

Third-party licenses
--------------------

42 changes: 42 additions & 0 deletions manual/kernel/pom.xml
@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>parent</artifactId>
<groupId>org.neo4j</groupId>
<version>3.0.7-SNAPSHOT</version>
<relativePath>../..</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>

<groupId>org.neo4j.doc</groupId>
<artifactId>neo4j-kernel-docs</artifactId>
<version>3.0.7-SNAPSHOT</version>

<name>Neo4j - Kernel Docs</name>
<description>Documentation build for the Neo4j kernel.</description>

<dependencies>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>${project.version}</version>
<scope>test</scope>
<type>test-jar</type>
</dependency>

</dependencies>
</project>
@@ -1,6 +1,5 @@
[[batchinsert]]
Batch Insertion
===============
= Batch Insertion

Neo4j has a batch insertion facility intended for initial imports, which bypasses transactions and other checks in favor of performance.
This is useful when you have a big dataset that needs to be loaded once.
Expand All @@ -13,53 +12,65 @@ Be aware of the following points when using batch insertion:
* Batch insertion is _not thread safe._
* Batch insertion is _non-transactional._
* Batch insertion is _not enforcing constraints_ on the inserted data while inserting data.
* Batch insertion will re-populate all existing indexes and indexes created during batch insertion on +shutdown+.
* Batch insertion will verify all existing constraints and constraints created during batch insertion on +shutdown+.
* Unless +shutdown+ is successfully invoked at the end of the import, the database files _will_ be corrupt.
* Batch insertion will re-populate all existing indexes and indexes created during batch insertion on `shutdown`.
* Batch insertion will verify all existing constraints and constraints created during batch insertion on `shutdown`.
* Unless `shutdown` is successfully invoked at the end of the import, the database files _will_ be corrupt.

[WARNING]
Always perform batch insertion in a _single thread_ (or use synchronization to make only one thread at a time access the batch inserter) and invoke +shutdown+ when finished.
--
Always perform batch insertion in a _single thread_ (or use synchronization to make only one thread at a time access the batch inserter) and invoke `shutdown` when finished.
--

[WARNING]
Since the batch insertion doesn't enforce constraint during data loading, if the inserted data violate any constraint the batch inserter will fail on +shutdown+ and the database _will_ be inconsistent.
--
Since the batch insertion doesn't enforce constraint during data loading, if the inserted data violate any constraint the batch inserter will fail on `shutdown` and the database _will_ be inconsistent.
--


[[batchinsert-examples]]
== Batch Inserter Examples ==
== Batch inserter examples


=== Initial import ===
=== Initial import

To bulk load data using the batch inserter you'll need to write a Java application which makes use of the low level link:javadocs/org/neo4j/unsafe/batchinsert/BatchInserter.html[+BatchInserter+] interface.
To bulk load data using the batch inserter you'll need to write a Java application which makes use of the low level link:javadocs/org/neo4j/unsafe/batchinsert/BatchInserter.html[`BatchInserter`] interface.

[TIP]
--
You can't have multiple threads using the batch inserter concurrently without external synchronization.
--

You can get hold of an instance of +BatchInserter+ by using link:javadocs/org/neo4j/unsafe/batchinsert/BatchInserters.html[+BatchInserters+].
You can get hold of an instance of `BatchInserter` by using link:javadocs/org/neo4j/unsafe/batchinsert/BatchInserters.html[`BatchInserters`].
Here's an example of the batch inserter in use:

[snippet,java]
[snippet, java]
----
component=neo4j-kernel
component=neo4j-kernel-docs
source=examples/BatchInsertDocTest.java
tag=insert
----

When creating a relationship you can set properties on the relationship by passing in a map containing properties rather than `null` as the last parameter to +createRelationship+.
When creating a relationship you can set properties on the relationship by passing in a map containing properties rather than `null` as the last parameter to `createRelationship`.

It's important that the call to `shutdown` is inside a finally block to ensure that it gets called even if exceptions are thrown.
If he batch inserter isn't cleanly shutdown then the consistency of the store is not guaranteed.

[TIP]
--
The source code for the examples on this page can be found here:
https://github.com/neo4j/neo4j/blob/{neo4j-git-tag}/community/kernel/src/test/java/examples/BatchInsertDocTest.java[BatchInsertDocTest.java]
--

=== Setting configuration options ===

You can pass custom configuration options to the +BatchInserter+. (See <<configuration-batchinsert>> for information on the available options.)
=== Setting configuration options

You can pass custom configuration options to the `BatchInserter`.
(See <<configuration-batchinsert>> for information on the available options.)
e.g.

[snippet,java]
[snippet, java]
----
component=neo4j-kernel
component=neo4j-kernel-docs
source=examples/BatchInsertDocTest.java
tag=configuredInsert
----
Expand All @@ -72,20 +83,23 @@ Alternatively you could store the configuration in a file:
include::../batchinsert-config[]
----

You can then refer to that file when initializing +BatchInserter+:
You can then refer to that file when initializing `BatchInserter`:

[snippet,java]
[snippet, java]
----
component=neo4j-kernel
component=neo4j-kernel-docs
source=examples/BatchInsertDocTest.java
tag=configFileInsert
----

=== Importing into an existing database ===

=== Importing into an existing database

Although it's a less common use case, the batch inserter can also be used to import data into an existing database.
However, you will need to ensure that the existing database is _shut down_ before you write to it.

[WARNING]
--
Since the batch importer bypasses transactions there is a possibility of data inconsistency if the import process crashes midway.
We would strongly suggest you take a backup of your existing database before using the batch inserter against it.
--
Expand Up @@ -181,23 +181,23 @@ In the following sections you'll find example code in Java which shows how this
[[transactions-deadlocks-template]]
==== Handling deadlocks using TransactionTemplate

If you don't want to write all the code yourself, there is a class called +link:javadocs/org/neo4j/helpers/TransactionTemplate.html[TransactionTemplate]+ that will help you achieve what's needed.
If you don't want to write all the code yourself, there is a class called link:javadocs/org/neo4j/helpers/TransactionTemplate.html[TransactionTemplate] that will help you achieve what's needed.
Below is an example of how to create, customize, and use this template for retries in transactions.

First, define the base template:

[snippet,java]
[snippet, java]
----
component=neo4j-kernel
component=neo4j-kernel-docs
source=examples/DeadlockDocTest.java
tag=template
----

Next, specify the database to use and a function to execute:

[snippet,java]
[snippet, java]
----
component=neo4j-kernel
component=neo4j-kernel-docs
source=examples/DeadlockDocTest.java
tag=usage-template
----
Expand All @@ -214,9 +214,9 @@ The template allows setting a predicate for what exceptions to retry on, and als
If you want to roll your own retry-loop code, see below for inspiration.
Here's an example of what a retry block might look like:

[snippet,java]
[snippet, java]
----
component=neo4j-kernel
component=neo4j-kernel-docs
source=examples/DeadlockDocTest.java
tag=retry
----
Expand Down
Expand Up @@ -5,17 +5,17 @@
* This file is part of Neo4j.
*
* Neo4j is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package examples;

Expand Down

0 comments on commit 4a87a77

Please sign in to comment.