Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion modules/ROOT/content-nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@ Generic Start

* **Import data**
** xref:import/introduction.adoc[What is Import?]
** xref:import/quick-start.adoc[Quick start]
** xref:import/visual-tour.adoc[Visual tour]
** xref:import/file-provision.adoc[File provision]
** xref:import/file-provision.adoc[Data provision]
** xref:import/modeling.adoc[Model data]
** xref:import/mapping.adoc[Map data]
** xref:import/indexes-and-constraints.adoc[Indexes and constraints]
Expand Down
Binary file modified modules/ROOT/images/connection-dropdown.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/ROOT/images/data-source.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified modules/ROOT/images/dropdown.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified modules/ROOT/images/file-filtering.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/ROOT/images/go-back.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/ROOT/images/import-ready.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified modules/ROOT/images/node-exclude.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified modules/ROOT/images/node-id.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified modules/ROOT/images/node-mapping.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/ROOT/images/sources.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added modules/ROOT/images/upx-import2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 15 additions & 8 deletions modules/ROOT/pages/import/file-provision.adoc
Original file line number Diff line number Diff line change
@@ -1,24 +1,31 @@
[[aura-file-provision]]
= File provision
= Data provision
:description: This section describes how to provide files for import.

In essence, you provide a set of files that contain the data to be imported and Import imports them into your instance.
You start by connecting your data source, which can be a relational database or local flat files.

Import supports flat files, i.e. files that contain data in a tabular format where each row represents a record and each column represents a field in that record.
The most common format for such files is CSV (comma-separated values), but Import also supports TSV (tab-separated values).
In essence, you provide the data in some format to be imported and Import imports this into your instance.

Import requires all files to have a header row and at least one row of data.
Import supports relational databases and flat files, i.e. files that contain data in a tabular format where each row represents a record and each column represents a field in that record.
The most common format for flat files is CSV (comma-separated values), but Import also supports TSV (tab-separated values).

When connecting to a remote data source, the tables are provided for you from that database.

But when you stream your local CSV files, the process can be more iterative and manual.

Import requires all CSV files to have a header row and at least one row of data.
The header row speficies how the data in the file should be interpreted and contains information for each field.
For more information about CSV files and the header format, see link:{neo4j-docs-base-uri}/operations-manual/current/tools/neo4j-admin/neo4j-admin-import/#import-tool-header-format[Operations Manual -> CSV header format].
Keep in mind that the column names must be unique, i.e. it is not possible to have two columns with the same name within the same file.

The files are provided in the _Files_ panel of Import.
The CSV files are provided in the _Data source_ panel of Import.
You can browse for them or drag and drop them into the panel.
Once a file is added to the panel, you can preview the header and the first row of data in the file by expanding the file.

[.shadow]
image::files.png[]

When you provide files to Import, they are temporarily stored on the client side until you run the import.
If you reload the page before running the import, the files are no longer available to the page and you need to provide them again.
When you provide CSV files to Import, only a reference to your local files is kept.
This is used to send the necessary commands to stream the data to your target database when you run the import.
Your local files themselves are *not* uploaded anywhere and therefore, if you reload the page before running the import, the files are no longer available to the page and you need to provide them again.
This is due to security features of your browser.
15 changes: 12 additions & 3 deletions modules/ROOT/pages/import/introduction.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,18 @@
= Import

The import service contains a tool for importing data into your instance and is ideal to get started quickly with testing and prototyping.
It allows you to import data from flat files without using any code.
This means that you can import data from any source that can be exported to a flat file and does thus not require DBMS filesystem access.
It allows you to import data without using any code from:

* PostgreSQL
* MySQL
* SQL Server
* .CSV

This service is also available as a standalone tool and is accessible at link:https://data-importer.neo4j.io/[] using a secure connection.
Additionally, Data Importer standalone is also available at link:https://data-importer.graphapp.io/[] with both secure and insecure connections.
For more information, see the link:https://neo4j.com/docs/data-importer/current/[Data Importer documentation].
For more information, see the link:https://neo4j.com/docs/data-importer/current/[Data Importer documentation].

[NOTE]
====
Only CSV files can be imported in the standalone tool.
====
8 changes: 4 additions & 4 deletions modules/ROOT/pages/import/mapping.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ Affected elements already imported are updated and **not** duplicated.

== Nodes

To map a node to a file, the node needs to have a label, which you can type directly on the node or in the mapping details panel.
After naming the label, you can select the file to map to the node.
Adding the file can be done at any time before running the import, but it is convenient to do it at this stage.
To map a node to a file/table, the node needs to have a label, which you can type directly on the node or in the mapping details panel.
After naming the label, you can select the file/table to map to the node.
If you are streaming local files, you can add the file at any time before running the import, but it is convenient to do it at this stage.

[.shadow]
image::node-mapping.png[width=400]
Expand Down Expand Up @@ -91,7 +91,7 @@ By default, Import excludes any rows where the value of the node ID column is em

The node exclude list is available from the more menu (`...`) in the data model panel, under _Settings_.

image::node-exclude.png[width=300]
image::node-exclude.png[width=400]


== Complete the mapping
Expand Down
8 changes: 5 additions & 3 deletions modules/ROOT/pages/import/modeling.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
= Data modeling

The data model is the is the blueprint for the database.
It defines how the data is organized and is the key to create a graph from your flat files.
The data model is what you map your flat files to.
It defines how the data is organized and is the key to create a graph from your files.
The data model is what you map your files to.
It consists of nodes and relationships.
Nodes need to have a _label_ and one or more _properties_.
Relationships need to have a _type_ and one or more _properties_.

//Add pointer to Graph Academy course on Data Modeling?
When you connect a remote data source, you are offered to have the model generated and mapped for you.
You can also upload a model, with or without data, in a _.json_ format.
But if you want full control of how your data is modeled, you can create the model manually.

The data model panel is located in the center of the screen.
It has buttons for collapsing the files panel, adding a node, discarding elements, previewing, running an import, and a more options button.
Expand Down
52 changes: 52 additions & 0 deletions modules/ROOT/pages/import/quick-start.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
[[quick-start]]
:description: This section gives an overview of the Import service.
= Quick start

The Import service consists of three tabs; *Data sources*, *Graph models*, and *Import jobs*.
If you haven't previously imported any data, all three are empty, otherwise sources, models, and import jobs are listed here.

[.shadow]
image::data-source.png[width=800]

== Provide the data

To get started you need to connect to a data source.
Import supports PostgreSQL, MySQL, SQL Server, as well as locally hosted flat files.

[.shadow]
image::sources.png[width=400]

For SQL-files, you need to configure the data source, add user credentials for the SQL-database, and give the data source a name.
If you want to stream local files, you can drag and drop them into the data source panel or browse for them.

== Model the data

When you have connected a data source, you have the option to have a model generated based on primary and foreign key constraints in the source database.
The quickest way is to accept to have a model generated, but you can draw your own later, see xref:import/modeling.adoc[] for more information.

If you use local files, you can upload a _.json_ file containing your model, if available, or draw your own.

If you generate a model from your relational data the model is already mapped to your tables.
This is indicated by a green checkmark in the model.
If you upload a local model, this may or may not be mapped already (it depends on whether it was mapped when first exported or not).
If any element is unmapped, the import will fail and you will be prompted to add the missing mapping information.
For more information on this, see xref:import/mapping.adoc[].
When all elements in the model are mapped, as indicated with the green checkmark, you can run the import.

[.shadow]
image::import-ready.png[width=400]

== Run the import

When you have connected a SQL data source, you need to provide credentials to the source and the destination (your Aura instance) in order to run the import.
However, if you are streaming local files, you just need to make sure that they are available in the data source panel and re-provide them if they are not.

If you have selected to go forward with a data source or an existing model and change your mind, click on *Graph models* to find the three tabs again.
From here, you can go back to *Data sources* to select a different source, if you want.

[.shadow]
image::go-back.png[width=300]

Connected data sources, models, and past import jobs are then listed under the respective tabs.

For more detailed information on xref:import/file-provision.adoc[file provision], xref:import/modeling[data models], and xref:import/import.adoc[import], see the respective pages.
20 changes: 12 additions & 8 deletions modules/ROOT/pages/import/visual-tour.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,19 @@
:description: This section provides an overview of the Import user interface.
= Visual tour

Once you have connected to a data source, Importer's interface is the same regardless of your data source.
It is divided into three parts, the data source panel where you provide the tables/files, the data model panel where you draw your model, and the details panel where you map your model to your tables.

[.shadow]
image::upx-import.png[]
image::upx-import2.png[]

== Files panel
== Data source panel

The source files for the import are placed here, either you drag and drop or browse for them.
Each file can be expanded using the dropdown option, to display the header and the first row of the file.
The `...` menu allows you to delete the file.
See xref:import/file-provision.adoc[] for more information about files.
The source tables for the import are placed here.
If you are using local CSV files, either you drag and drop them here or browse for them.
Each table can be expanded using the dropdown option, to display the header and the first row of the table.
The `...` menu allows you to delete the table.
See xref:import/file-provision.adoc[] for more information about files and tables.

== Data model panel

Expand All @@ -25,7 +29,7 @@ See xref:import/modeling.adoc[] about how to create the data model.

== Delete selection

Select one or more elements on the canvaas and then use this button to delete them.
Select one or more elements on the canvas and then use this button to delete them.

== More menu

Expand All @@ -35,7 +39,7 @@ For more information on the settings, see xref:import/mapping.adoc#exclude-list[
== Details panel

The details panel contains two tabs, _Definition_ and _Constraints & Indexes_.
The _Defintion_ tab is where you give labels to nodes and relationships, map them to files from the files panel, and specify their properties.
The _Defintion_ tab is where you give labels to nodes and relationships, map them to files from the data source panel, and specify their properties.
See xref:import/mapping.adoc[] for more information.
The _Constraints & Indexes_ tab allows you to see details about constraints and indexes and lets you add or modify the latter.
See xref:import/indexes-and-constraints.adoc[] for more information.