-
Notifications
You must be signed in to change notification settings - Fork 27
Import updates #554
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Import updates #554
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
e7d335e
wip on content
AlexicaWright fdf93e0
add images and new stuff and updates
AlexicaWright d56a2dd
more images and updated content
AlexicaWright 1957dbd
correction
AlexicaWright 89fc98b
addressed pr comments
AlexicaWright 449a6a0
updates
AlexicaWright File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,24 +1,31 @@ | ||
[[aura-file-provision]] | ||
= File provision | ||
= Data provision | ||
:description: This section describes how to provide files for import. | ||
|
||
In essence, you provide a set of files that contain the data to be imported and Import imports them into your instance. | ||
You start by connecting your data source, which can be a relational database or local flat files. | ||
|
||
Import supports flat files, i.e. files that contain data in a tabular format where each row represents a record and each column represents a field in that record. | ||
The most common format for such files is CSV (comma-separated values), but Import also supports TSV (tab-separated values). | ||
In essence, you provide the data in some format to be imported and Import imports this into your instance. | ||
|
||
Import requires all files to have a header row and at least one row of data. | ||
Import supports relational databases and flat files, i.e. files that contain data in a tabular format where each row represents a record and each column represents a field in that record. | ||
The most common format for flat files is CSV (comma-separated values), but Import also supports TSV (tab-separated values). | ||
|
||
When connecting to a remote data source, the tables are provided for you from that database. | ||
|
||
But when you stream your local CSV files, the process can be more iterative and manual. | ||
|
||
Import requires all CSV files to have a header row and at least one row of data. | ||
The header row speficies how the data in the file should be interpreted and contains information for each field. | ||
For more information about CSV files and the header format, see link:{neo4j-docs-base-uri}/operations-manual/current/tools/neo4j-admin/neo4j-admin-import/#import-tool-header-format[Operations Manual -> CSV header format]. | ||
Keep in mind that the column names must be unique, i.e. it is not possible to have two columns with the same name within the same file. | ||
|
||
The files are provided in the _Files_ panel of Import. | ||
The CSV files are provided in the _Data source_ panel of Import. | ||
You can browse for them or drag and drop them into the panel. | ||
Once a file is added to the panel, you can preview the header and the first row of data in the file by expanding the file. | ||
|
||
[.shadow] | ||
image::files.png[] | ||
|
||
When you provide files to Import, they are temporarily stored on the client side until you run the import. | ||
If you reload the page before running the import, the files are no longer available to the page and you need to provide them again. | ||
When you provide CSV files to Import, only a reference to your local files is kept. | ||
This is used to send the necessary commands to stream the data to your target database when you run the import. | ||
Your local files themselves are *not* uploaded anywhere and therefore, if you reload the page before running the import, the files are no longer available to the page and you need to provide them again. | ||
This is due to security features of your browser. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
[[quick-start]] | ||
:description: This section gives an overview of the Import service. | ||
= Quick start | ||
|
||
The Import service consists of three tabs; *Data sources*, *Graph models*, and *Import jobs*. | ||
If you haven't previously imported any data, all three are empty, otherwise sources, models, and import jobs are listed here. | ||
|
||
[.shadow] | ||
image::data-source.png[width=800] | ||
|
||
== Provide the data | ||
|
||
To get started you need to connect to a data source. | ||
Import supports PostgreSQL, MySQL, SQL Server, as well as locally hosted flat files. | ||
|
||
[.shadow] | ||
image::sources.png[width=400] | ||
|
||
For SQL-files, you need to configure the data source, add user credentials for the SQL-database, and give the data source a name. | ||
If you want to stream local files, you can drag and drop them into the data source panel or browse for them. | ||
|
||
== Model the data | ||
|
||
When you have connected a data source, you have the option to have a model generated based on primary and foreign key constraints in the source database. | ||
The quickest way is to accept to have a model generated, but you can draw your own later, see xref:import/modeling.adoc[] for more information. | ||
|
||
If you use local files, you can upload a _.json_ file containing your model, if available, or draw your own. | ||
|
||
If you generate a model from your relational data the model is already mapped to your tables. | ||
This is indicated by a green checkmark in the model. | ||
If you upload a local model, this may or may not be mapped already (it depends on whether it was mapped when first exported or not). | ||
If any element is unmapped, the import will fail and you will be prompted to add the missing mapping information. | ||
For more information on this, see xref:import/mapping.adoc[]. | ||
When all elements in the model are mapped, as indicated with the green checkmark, you can run the import. | ||
|
||
[.shadow] | ||
image::import-ready.png[width=400] | ||
|
||
== Run the import | ||
|
||
When you have connected a SQL data source, you need to provide credentials to the source and the destination (your Aura instance) in order to run the import. | ||
However, if you are streaming local files, you just need to make sure that they are available in the data source panel and re-provide them if they are not. | ||
|
||
If you have selected to go forward with a data source or an existing model and change your mind, click on *Graph models* to find the three tabs again. | ||
From here, you can go back to *Data sources* to select a different source, if you want. | ||
|
||
[.shadow] | ||
image::go-back.png[width=300] | ||
|
||
Connected data sources, models, and past import jobs are then listed under the respective tabs. | ||
|
||
For more detailed information on xref:import/file-provision.adoc[file provision], xref:import/modeling[data models], and xref:import/import.adoc[import], see the respective pages. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.