Skip to content

Commit

Permalink
0004097: Router convertToReload that converts multiple changes into a
Browse files Browse the repository at this point in the history
reload batch
  • Loading branch information
erilong committed Sep 23, 2019
1 parent 942d9e1 commit c58da50
Show file tree
Hide file tree
Showing 3 changed files with 42 additions and 10 deletions.
21 changes: 11 additions & 10 deletions symmetric-assemble/src/asciidoc/configuration/routers.ad
Expand Up @@ -18,26 +18,27 @@ Router Type:: The type of router. Standard router types are listed below. Cust
[cols="<2,^1,<9", options="header"]
|===
|Type
|Provided
|Description

|default|x|A router that sends all captured data to all nodes that belong to the target node group defined in the router. See <<Default Router>>
|default|A router that sends all captured data to all nodes that belong to the target node group defined in the router. See <<Default Router>>

|column|x|A router that compares old or new column values in a captured data row to a constant value or the value of a target node's external id or node id. See <<Column Match Router>>
|column|A router that compares old or new column values in a captured data row to a constant value or the value of a target node's external id or node id. See <<Column Match Router>>

|audit|x|A router that inserts into an automatically created audit table. It records captured changes to tables that it is linked to. See <<Audit Table Router>>
|audit|A router that inserts into an automatically created audit table. It records captured changes to tables that it is linked to. See <<Audit Table Router>>

|java|x|A router that executes a Java expression in order to select nodes to route to. The script can use the old and new column values. See <<Java Router>>
|java|A router that executes a Java expression in order to select nodes to route to. The script can use the old and new column values. See <<Java Router>>

|lookuptable|x|A router which can be configured to determine routing based on an existing or ancillary table specifically for the purpose of routing data. See <<Lookup Table Router>>
|lookuptable|A router which can be configured to determine routing based on an existing or ancillary table specifically for the purpose of routing data. See <<Lookup Table Router>>

|subselect|x|A router that executes a SQL expression against the database to select nodes to route to. This SQL expression can be passed values of old and new column values. See <<Subselect Router>>
|subselect|A router that executes a SQL expression against the database to select nodes to route to. This SQL expression can be passed values of old and new column values. See <<Subselect Router>>

|bsh|x|A router that executes a Bean Shell script expression in order to select nodes to route to. The script can use the old and new column values. See <<Beanshell Router>>
|convertToReload|When subselect router is too slow and there are too many tables or rows for lookuptable router, a convertToReload router can efficiently sub-set data by converting multiple changes into a reload batch. See <<ConvertToReload Router>>

|csv|x|A router that sends data from a CSV file to a target table of the target node group defined in the router. See <<CSV Router>>
|bsh|A router that executes a Bean Shell script expression in order to select nodes to route to. The script can use the old and new column values. See <<Beanshell Router>>

|dbf|x|A router that sends data from a dBase generated DBF file to a target table of the target node group defined in the router. See <<DBF Router>>
|csv|A router that sends data from a CSV file to a target table of the target node group defined in the router. See <<CSV Router>>

|dbf|A router that sends data from a dBase generated DBF file to a target table of the target node group defined in the router. See <<DBF Router>>

|===

Expand Down
@@ -0,0 +1,30 @@

===== ConvertToReload Router

This router converts multiple change events into a single reload event, which can be used to sub-set data quickly
when the subselect router is too slow or when the lookuptable router can't handle the amount of lookup data.
ConvertToReload inserts the primary key values for each row, along with a unique load ID, into a temporary table.
The reload event uses the initial load SQL along with a join to the temporary table to retrieve the changes
and sub-set data.

The router expression requires the name of the temporary table to use:

----
temptable=mytable_router
----

.Sample Use Case for ConvertToReload Router
====

[source, SQL]
----
create table mytest (id integer, name varchar(50), primary key(id));
create table mytest_router (id integer, load_id integer, primary key(load_id, id));
insert into SYM_ROUTER (router_id,
source_node_group_id, target_node_group_id, router_type,
router_expression, create_time, last_update_time) values
('corp-2-store','corp', 'store', 'convertToReload', 'temptable=mytest_router', current_timestamp,
current_timestamp);
----

====
Expand Up @@ -6,6 +6,7 @@ include::column.ad[]
include::audit.ad[]
include::lookuptable.ad[]
include::subselect.ad[]
include::convertToReload.ad[]
include::bsh.ad[]
include::csv.ad[]
include::dbf.ad[]

0 comments on commit c58da50

Please sign in to comment.