0005344: Documentation for Bulk Loading

JumpMind · Jul 7, 2022 · 472dbd1 · 472dbd1
1 parent 6783a55
commit 472dbd1
Show file tree

Hide file tree

Showing 6 changed files with 37 additions and 15 deletions.
diff --git a/symmetric-assemble/src/asciidoc/appendix/databases.ad b/symmetric-assemble/src/asciidoc/appendix/databases.ad
@@ -265,8 +265,8 @@ by database.
 |✔
 |✔
 |
-|?
-|?
+|✔
+|✔
 
 |PostgreSQL
 |8.2.5 and above

diff --git a/symmetric-assemble/src/asciidoc/appendix/mariadb.ad b/symmetric-assemble/src/asciidoc/appendix/mariadb.ad
@@ -25,13 +25,13 @@ See MySQL notes.  You can use either the MySQL or MariaDB driver for this dialec
 |===
 
 ==== Loading Data
-When registering a Maria database with Symmetric, bulk loading will be checked by default.
+When registering a Maria database with SymmetricDS, bulk loading will be checked by default.
 
 image::images/appendix/mysql-enable-bulk-loading.png[]
 
-To succesfully bulk load, once your MySQL database is registered with Symmetric, go to the top of the screen and select the MySQL node you want to bulk load from the drop down list of your nodes.
+To succesfully bulk load, once your MySQL database is registered with SymmetricDS, go to the top of the screen and select the MySQL node you want to bulk load from the drop down list of your nodes.
 
-Then, go to the Manage tab, and go to Startup Parameters and set mysql.bulk.load.local to true, and restart Symmetric.
+Then, go to the Manage tab, and go to Startup Parameters and set mysql.bulk.load.local to true, and restart SymmetricDS.
 
 image::images/appendix/mysql-bulk-load-local-true.png[]
 

diff --git a/symmetric-assemble/src/asciidoc/appendix/mssqlserver.ad b/symmetric-assemble/src/asciidoc/appendix/mssqlserver.ad
@@ -124,4 +124,4 @@ endif::pro[]
 
 ==== Loading Data
 
-When registering a SQL Server database with Symmetric, bulk loading will be be checked by default. In order to make use of bulk loading, the user must provide a path to a bcp executable (installation for this executable is linked in the Add Database Wizard when asked about bulk loading).
+When registering a SQL Server database with SymmetricDS, bulk loading will be be checked by default. In order to make use of bulk loading, the user must provide a path to a bcp executable (installation for this executable is linked in the Add Database Wizard when asked about bulk loading).
diff --git a/symmetric-assemble/src/asciidoc/appendix/mysql.ad b/symmetric-assemble/src/asciidoc/appendix/mysql.ad
@@ -98,16 +98,16 @@ endif::pro[]
 |===
 
 ==== Loading Data
-When registering a MySQL database with Symmetric, bulk loading will be checked by default.
+When registering a MySQL database with SymmetricDS, bulk loading will be checked by default.
 
 image::images/appendix/mysql-enable-bulk-loading.png[]
 
-To succesfully bulk load, once your MySQL database is registered with Symmetric, go to the top of the screen and select the MySQL node you want to bulk load from the drop down list of your nodes.
+To succesfully bulk load, once your MySQL database is registered with SymmetricDS, go to the top of the screen and select the MySQL node you want to bulk load from the drop down list of your nodes.
 
 image::images/appendix/mysql-selecting-node-1.png[]
 image::images/appendix/mysql-selecting-node-2.png[]
 
-Then, go to the Manage tab, and go to Startup Parameters and set mysql.bulk.load.local to true, and restart Symmetric.
+Then, go to the Manage tab, and go to Startup Parameters and set mysql.bulk.load.local to true, and restart SymmetricDS.
 
 image::images/appendix/mysql-bulk-load-local-true.png[]
 

diff --git a/symmetric-assemble/src/asciidoc/appendix/oracle.ad b/symmetric-assemble/src/asciidoc/appendix/oracle.ad
@@ -26,7 +26,7 @@ alter database add supplemental log data (all) columns;
 If using Oracle 11g without CDB, use the following permissions:
 
 [source, SQL]
-grant create session, alter session, select any dictionary, execute_catalog_role to symmetric;
+grant create session, alter session, select any dictionary, select any transaction, execute_catalog_role to symmetric;
 
 If using Oracle 12c without CDB, use the following permissions:
 
@@ -52,7 +52,30 @@ sym_data is cast to a clob which prevents truncation at 4k.  There is overhead f
 
 ==== Loading Data
 SymmetricDS has bulk loading capability available for Oracle. SymmetricDS specifies data loader types on a channel by channel basis. 
-To utilize Oracle Bulk loading versus straight JDBC insert, specify the Oracle Bulk Loader ("bulk") in the data_loader_type column of sym_channel.
+
+To utilize bulk loading with Oracle, you should create a user that is not the default system user of the database. Whatever user you end up using, make sure it also has the permissions mentioned above in the Database Permission section, as well as the ones mentioned below. It's important to note that SQL Loader (what Oracle uses to bulk load) will not work if either your user name or password contains a '@' symbol, so make sure to avoid this in both your username and password.
+
+----
+create user yourusername identified by "yourpassword";
+----
+
+Once your user is created, make sure to add the following permissions to the user to make sure that Symmetric has the correct permissions to bulk load with your database. (Again, make sure your user has the permissions mentioned in the Database Permission section as well)
+
+----
+GRANT CREATE SESSION TO yourusername;
+GRANT CREATE TABLE TO yourusername;
+alter user yourusername quota unlimited on system;
+GRANT CONNECT TO yourusername;
+GRANT RESOURCE TO yourusername;
+GRANT CREATE ANY TRIGGER TO yourusername;
+GRANT EXECUTE ON UTL_RAW To yourusername;
+----
+
+Register your Oracle database with Symmetric using the user you've just created. Once you have done that, when you get to the Bulk Loading screen on the Add Database Wizard you will need to provide a path to your SQL Loader installation. If you do not have SQL Loader installed, there is a link provided in the Add Database Wizard where you can go to get this installed.
+
+Once you have your database registered, to utilize Oracle Bulk loading versus straight JDBC insert, specify the Oracle Bulk Loader ("bulk") in the data_loader_type column of sym_channel table.
+
+Once these steps are complete your database should be configured to bulk load with SymmetricDS.
 
 ==== Optional - Partitioning
 Partitioning the DATA table by channel can help insert, routing and extraction performance on concurrent, high throughput systems. 

diff --git a/symmetric-assemble/src/asciidoc/appendix/postgresql.ad b/symmetric-assemble/src/asciidoc/appendix/postgresql.ad
@@ -1,9 +1,6 @@
 
 === PostgreSQL
 
-SymmetricDS has bulk loading capability available for Postgres. SymmetricDS specifies data loader types on a channel by channel basis.
-To utilize Postgres Bulk loading versus straight JDBC insert, specify the Postgres Bulk Loader ("postgres_bulk") in the data_loader_type column of sym_channel.
-
 Starting with PostgreSQL 8.3, SymmetricDS supports the transaction identifier.
 Binary Large Object (BLOB) replication is supported for both byte array (BYTEA) and object ID (OID) data types.
 
@@ -88,4 +85,6 @@ endif::pro[]
 |===
 
 ==== Loading Data
-When registering a PostgreSQL database with Symmetric, bulk loading will be checked by default. If you wish to bulk load with PostgreSQL, all the user must do is leave this checked upon setup.
+
+SymmetricDS has bulk loading capability available for Postgres, when registering a PostgreSQL database with SymmetricDS, bulk loading will be checked by default. SymmetricDS specifies data loader types on a channel by channel basis.
+To utilize Postgres Bulk loading versus straight JDBC insert, specify the Postgres Bulk Loader ("postgres_bulk") in the data_loader_type column of sym_channel.