Skip to content

Commit

Permalink
#18 Implement one configuration file
Browse files Browse the repository at this point in the history
  • Loading branch information
@IlyaLisov
IlyaLisov committed Jun 26, 2023
1 parent 0eb3bc8 commit 8886210
Show file tree
Hide file tree
Showing 8 changed files with 177 additions and 172 deletions.
2 changes: 1 addition & 1 deletion .env
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ NEO4J_DATABASE=testneo4j
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=password

XML_CONFIG_LOCATION=src/main/resources/xml/node-script.xml
XML_CONFIG_LOCATION=src/main/resources/xml/script.xml
133 changes: 68 additions & 65 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,81 +27,84 @@ Future features:
Here is an example of XML configuration file for node and relationship
migration.

You can validate your schema with `node-schema.xsd`
and `relationship-schema.xsd` schemas.
You can validate your schema with `schema.xsd`schema.

```
<migration type="node">
<tables>
<table name="users">
<configuration>
<excludedColumns>
<column>surname</column>
</excludedColumns>
<renamedColumns>
<columns>
<previousName>name</previousName>
<newName>newName</newName>
</columns>
</renamedColumns>
</configuration>
<labels>
<label>User</label>
<label>BaseEntity</label>
</labels>
</table>
<table name="tasks">
<labels>
<label>Task</label>
<label>BaseEntity</label>
</labels>
</table>
</tables>
<migration>
<node>
<tables>
<table name="users">
<configuration>
<excludedColumns>
<column>surname</column>
</excludedColumns>
<renamedColumns>
<columns>
<previousName>name</previousName>
<newName>newName</newName>
</columns>
</renamedColumns>
</configuration>
<labels>
<label>User</label>
<label>BaseEntity</label>
</labels>
</table>
<table name="tasks">
<labels>
<label>Task</label>
<label>BaseEntity</label>
</labels>
</table>
</tables>
</node>
<relationship>
<tables>
<table name="users_tasks">
<configuration>
<columnFrom>user_id</columnFrom>
<labelFrom>User</labelFrom>
<columnTo>task_id</columnTo>
<labelTo>Task</labelTo>
</configuration>
<type>HAS_TASK</type>
</table>
</tables>
</relationship>
</migration>
```

```
<migration type="relationship">
<tables>
<table name="users_tasks">
<configuration>
<columnFrom>user_id</columnFrom>
<labelFrom>User</labelFrom>
<columnTo>task_id</columnTo>
<labelTo>Task</labelTo>
</configuration>
<type>HAS_TASK</label>
</table>
</tables>
</migration>
```

1) `<migration>` - main tag, defines whether Node or Relationship is going to be
created. You need to provide `type` attribute - `node` or `relationship`.
2) `<tables>` - collection of tables to be migrated.
3) `<table>` - table tag, defines table name in `name` attribute, its
1) `<migration>` - main tag, contains what Node or/and Relationship is going to
be created.
2) `<node>` - (optional) node migration description tag.
3) `<relationship>` - (optional) relationship migration description tag.
4) `<tables>` - collection of tables to be migrated.
5) `<table>` - table tag, defines table name in `name` attribute, its
configuration and labels.
4) `<configuration>` - (optional for `node` migration mode) configuration for
6) `<configuration>` - (optional for `node` migration) configuration for
columns.
5) `<excludedColumns>` - (optional) columns to be excluded from migration. It
means after
migration in Neo4j no data from these columns will be stored.
6) `<column>` - column tag, contains table name.
7) `<renamedColumns>` - (optional) columns to be renamed during migration. It
means data
from column with `<previousName>`
7) `<excludedColumns>` - (optional for `node` migration) columns to be excluded
from migration. It
means after migration in Neo4j no data from these columns will be stored.
8) `<column>` - column tag, contains table name.
9) `<renamedColumns>` - (optional for `node` migration) columns to be renamed
during migration. It means data from column with `<previousName>`
will be stored as `<newName>` property;
8) `<labels>` - (optional) collection of labels to be added to Nodes.
9) `<label>` - label tag, defines its name.
10) `<columnFrom>` - column with foreign key to entity table. Relationship will
10) `<labels>` - (optional for `node` migration) collection of labels to be
added
to Nodes.
11) `<label>` - label tag, defines its name.
12) `<columnFrom>` - column with foreign key to entity table. Relationship will
be started from Node from that table by this foreign key.
11) `<columnTo>` - column with foreign key to entity table. Relationship will
13) `<columnTo>` - column with foreign key to entity table. Relationship will
be ended with Node from that table by this foreign key.
12) `<labelFrom>` - (optional) specifies label of start node to find it by
14) `<labelFrom>` - (optional for `migration` mode) specifies label of start
node to find it by
foreign key.
12) `<labelTo>` - (optional) specifies label of end node to find it by foreign
15) `<labelTo>` - (optional for `migration` mode) specifies label of end node to
find it by foreign
key.
13) `<type>` - type of the relationship.
16) `<type>` - type of the relationship.

### NOTE

Expand All @@ -117,7 +120,7 @@ Note that at first we exclude columns and only after rename them. So if you will
rename excluded columns, it was excluded and no columns with this name will be
renamed.

We recommend to provide all available fields to be sure that correct data will
We recommend to fill up all tags to be sure that correct data will
be saved to Neo4j.

### How it works?
Expand All @@ -135,4 +138,4 @@ So if some of your nodes have similar id, relationship will be added to each of
them. It can be avoided but providing `<labelFrom>` and `<labelTo>` tags.

If no exceptions were thrown, you will see messages in logs with amount of
created nodes.
created nodes and relationships.
32 changes: 20 additions & 12 deletions src/main/java/com/example/postgresneo4jmigrationtool/parser/XmlParser.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,26 +33,34 @@ public class XmlParser implements Parser {
public void parse() {
XML root = xml.nodes("migration").get(0);
try {
MigrationType type = MigrationType.valueOf(root.xpath("@type").get(0).toUpperCase());
List<XML> tablesTag = root.nodes("tables");
if (tablesTag.isEmpty()) {
throw new InvalidConfigurationException("You must provide <tables> tag.");
}
List<XML> tables = tablesTag.get(0)
.nodes("table");
if (tables.isEmpty()) {
throw new InvalidConfigurationException("You must provide at least one <table> tag.");
}
if (type == MigrationType.NODE) {
List<XML> nodeMigration = root.nodes("node");
List<XML> relationshipMigration = root.nodes("relationship");
if (!nodeMigration.isEmpty()) {
List<XML> tables = getXMLTables(nodeMigration.get(0));
parseNodeMigration(tables);
} else if (type == MigrationType.RELATIONSHIP) {
}
if (!relationshipMigration.isEmpty()) {
List<XML> tables = getXMLTables(relationshipMigration.get(0));
parseRelationshipMigration(tables);
}
} catch (IllegalArgumentException e) {
throw new InvalidConfigurationException("Unsupported migration type. Supported are: " + Arrays.stream(MigrationType.values()).map(s -> s.name().toLowerCase()).toList());
}
}

private List<XML> getXMLTables(XML root) {
List<XML> tablesTag = root.nodes("tables");
if (tablesTag.isEmpty()) {
throw new InvalidConfigurationException("You must provide <tables> tag.");
}
List<XML> tables = tablesTag.get(0)
.nodes("table");
if (tables.isEmpty()) {
throw new InvalidConfigurationException("You must provide at least one <table> tag.");
}
return tables;
}

private void parseNodeMigration(List<XML> tables) {
for (XML table : tables) {
String tableName = getTableName(table);
Expand Down
27 changes: 0 additions & 27 deletions src/main/resources/xml/node-script.xml
View file

This file was deleted.

46 changes: 0 additions & 46 deletions src/main/resources/xml/relationship-schema.xsd
View file

This file was deleted.

13 changes: 0 additions & 13 deletions src/main/resources/xml/relationship-script.xml
View file

This file was deleted.

54 changes: 46 additions & 8 deletions src/main/resources/xml/node-schema.xsd → src/main/resources/xml/schema.xsd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,20 @@
<xs:element name="migration">
<xs:complexType>
<xs:sequence>
<xs:element name="tables" type="tablesType"/>
<xs:element name="node" type="nodeMigration"
minOccurs="0"/>
<xs:element name="relationship"
type="relationshipMigration" minOccurs="0"/>
</xs:sequence>
<xs:attribute name="type" use="required">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="node"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>

<xs:complexType name="nodeMigration">
<xs:sequence>
<xs:element name="tables" type="tablesType"/>
</xs:sequence>
</xs:complexType>

<xs:complexType name="tablesType">
<xs:sequence>
<xs:element name="table" maxOccurs="unbounded"
Expand Down Expand Up @@ -67,4 +69,40 @@
</xs:sequence>
</xs:complexType>

<xs:complexType name="relationshipMigration">
<xs:sequence>
<xs:element name="tables"
type="relationshipTables"/>
</xs:sequence>
</xs:complexType>

<xs:complexType name="relationshipTables">
<xs:sequence>
<xs:element name="table" maxOccurs="unbounded"
type="relationshipTableType"/>
</xs:sequence>
</xs:complexType>

<xs:complexType name="relationshipTableType">
<xs:sequence>
<xs:element name="configuration"
type="relationshipConfigurationType"
minOccurs="0"/>
<xs:element name="type" type="xs:string"
/>
</xs:sequence>
<xs:attribute name="name" type="xs:string" use="required"/>
</xs:complexType>

<xs:complexType name="relationshipConfigurationType">
<xs:sequence>
<xs:element name="columnFrom" type="xs:string"/>
<xs:element name="labelFrom" type="xs:string"
minOccurs="0"/>
<xs:element name="columnTo" type="xs:string"/>
<xs:element name="labelTo" type="xs:string"
minOccurs="0"/>
</xs:sequence>
</xs:complexType>

</xs:schema>
Loading

0 comments on commit 8886210

Please sign in to comment.