-
Notifications
You must be signed in to change notification settings - Fork 221
/
developer.ad
323 lines (238 loc) · 17 KB
/
developer.ad
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
== Developer
This chapter focuses on a variety of ways for developers to build upon and extend some of the existing features found within SymmetricDS.
=== Extension Points
SymmetricDS has a pluggable architecture that can be extended. A Java class that implements the appropriate extension point interface, can implement custom logic and change the behavior of SymmetricDS to suit special needs. All supported extension points extend the IExtensionPoint interface. The available extension points are documented in the following sections.
When SymmetricDS starts up, the ExtensionPointManager searches a Spring Framework context for classes that implement the IExtensionPoint interface, then creates and registers the class with the appropriate SymmetricDS component.
Extensions should be configured in the conf/symmetric-extensions.xml file as Spring beans. The jar file that contains the extension should be placed in the web/WEB-INF/lib directory.
If an extension point needs access to SymmetricDS services or needs to connect to the database it may implement the ISymmetricEngineAware interface in order to get a handle to the ISymmetricEngine.
The INodeGroupExtensionPoint interface may be optionally implemented to indicate that a registered extension point should only be registered with specific node groups.
[source, java]
----
/**
* Only apply this extension point to the 'root' node group.
*/
public String[] getNodeGroupIdsToApplyTo() {
return new String[] { "root" };
}
----
===== IParameterFilter
Parameter values can be specified in code using a parameter filter. Note that there can be only one parameter filter per engine instance. The IParameterFilter replaces the deprecated IRuntimeConfig from prior releases.
[source, java]
----
public class MyParameterFilter
implements IParameterFilter, INodeGroupExtensionPoint {
/**
* Only apply this filter to stores
*/
public String[] getNodeGroupIdsToApplyTo() {
return new String[] { "store" };
}
public String filterParameter(String key, String value) {
// look up a store number from an already existing properties file.
if (key.equals(ParameterConstants.EXTERNAL_ID)) {
return StoreProperties.getStoreProperties().
getProperty(StoreProperties.STORE_NUMBER);
}
return value;
}
public boolean isAutoRegister() {
return true;
}
}
----
===== IDatabaseWriterFilter
Data can be filtered or manipulated before it is loaded into the target database. A filter can change the data in a column, save it somewhere else or do something else with the data entirely. It can also specify by the return value of the function call that the data loader should continue on and load the data (by returning true) or ignore it (by returning false). One possible use of the filter, for example, might be to route credit card data to a secure database and blank it out as it loads into a less-restricted reporting database.
A DataContext is passed to each of the callback methods. A new context is created for each synchronization. The context provides a mechanism to share data during the load of a batch between different rows of data that are committed in a single database transaction.
The filter also provides callback methods for the batch lifecycle. The DatabaseWriterFilterAdapter may be used if not all methods are required.
A class implementing the IDatabaseWriterFilter interface is injected onto the DataLoaderService in order to receive callbacks when data is inserted, updated, or deleted.
[source, java]
----
public class MyFilter extends DatabaseWriterFilterAdapter {
@Override
public boolean beforeWrite(DataContext context, Table table, CsvData data) {
if (table.getName().equalsIgnoreCase("CREDIT_CARD_TENDER")
&& data.getDataEventType().equals(DataEventType.INSERT)) {
String[] parsedData = data.getParsedData(CsvData.ROW_DATA);
// blank out credit card number
parsedData[table.getColumnIndex("CREDIT_CARD_NUMBER")] = null;
}
return true;
}
}
----
The filter class should be specified in conf/symmetric-extensions.xml as follows.
[source, xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd">
<bean id="myFilter" class="com.mydomain.MyFilter"/>
</beans>
----
===== IDatabaseWriterErrorHandler
Implement this extension point to override how errors are handled. You can use this extension point to ignore rows that produce foreign key errors.
===== IDataLoaderFactory
Implement this extension point to provide a different implementation of the org.jumpmind.symmetric.io.data.IDataWriter that is used by the SymmetricDS data loader. Data loaders are configured for a channel. After this extension point is registered it can be activated for a CHANNEL by indicating the data loader name in the data_loader_type column.
SymmetricDS has two out of the box extensions of IDataLoaderFactory already implemented in its PostgresBulkDataLoaderFactory and OracleBulkDataLoaderFactory classes. These extension points implement bulk data loading capabilities for Oracle, Postgres and Greenplum dialects. See Appendix C. Database Notes for details.
Another possible use of this extension point is to route data to a NOSQL data sink.
===== IAcknowledgeEventListener
Implement this extension point to receive callback events when a batch is acknowledged. The callback for this listener happens at the point of extraction.
===== IReloadListener
Implement this extension point to listen in and take action before or after a reload is requested for a Node. The callback for this listener happens at the point of extraction.
===== ISyncUrlExtension
This extension point is used to select an appropriate URL based on the URI provided in the sync_url column of sym_node.
To use this extension point configure the sync_url for a node with the protocol of ext://beanName. The beanName is the name you give the extension point in the extension xml file.
===== IColumnTransform
This extension point allows custom column transformations to be created. There are a handful of out-of-the-box implementations. If any of these do not meet the column transformation needs of the application, then a custom transform can be created and registered. It can be activated by referencing the column transform's name transform_type column of TRANSFORM_COLUMN
===== INodeIdCreator
This extension point allows SymmetricDS users to implement their own algorithms for how node ids and passwords are generated or selected during the registration process. There may be only one node creator per SymmetricDS instance (Please note that the node creator extension has replaced the node generator extension).
===== ITriggerCreationListener
Implement this extension point to get status callbacks during trigger creation.
===== IBatchAlgorithm
Implement this extension point and set the name of the Spring bean on the batch_algorithm column of the Channel table to use. This extension point gives fine grained control over how a channel is batched.
===== IDataRouter
Implement this extension point and set the name of the Spring bean on the router_type column of the Router table to use. This extension point gives the ability to programmatically decide which nodes data should be routed to.
===== IHeartbeatListener
Implement this extension point to get callbacks during the heartbeat job.
===== IOfflineClientListener
Implement this extension point to get callbacks for offline events on client nodes.
===== IOfflineServerListener
Implement this extension point to get callbacks for offline events detected on a server node during monitoring of client nodes.
===== INodePasswordFilter
Implement this extension point to intercept the saving and rendering of the node password.
=== Embedding in Android
SymmetricDS has its web-enabled, fault-tolerant, database synchronization software available on the Android mobile computing platform.
The Android client follows all of the same concepts and brings to Android all of the same core SymmetricDS features as the full-featured,
Java-based SymmetricDS client. The Android client is a little bit different in that it is not a stand-alone application, but is designed to
be referenced as a library to run in-process with an Android application requiring synchronization for its SQLite database.
By using SymmetricDS, mobile application development is simplified, in that the mobile application developer can now focus solely on interacting
with their local SQLite database. SymmetricDS takes care of capturing and moving data changes to and from a centralized database when the
network is available
The same core libraries that are used for the SymmetricDS server are also used for Android. SymmetricDS's overall footprint is reduced by
eliminating a number of external dependencies in order to fit better on an Android device. The database access layer is abstracted so that
the Android specific database access layer could be used. This allows SymmetricDS to be efficient in accessing the SQLite database on the
Android device.
In order to convey how to use the SymmetricDS Android libraries, the example below will show how to integrate an Android application with the
Quick-Start demo configuration. This Android application will create an embedded SQLite database, all SymmetricDS run-time tables, and
configure the Quick-Start demo tables (ITEM, ITEM_SELLING_PRICE, SALE_TRANSACTION, SALE_RETURN_LINE_ITEM) for synchronization. The application
has a basic user interface that can run queries on the SQLite database to demonstrate synchronization with a Quick-Start demo corp server.
Android Studio 3.0.1 and Android SDK 26 were used for this example. The example Android application can be cloned into Android Studio using
the GitHub project located here: https://github.com/JumpMind/symmetric-android-client-demo
SymmetricDS for Android comes as a zip file of Java archives (jar files) that are required by the SymmetricDS client at runtime. The libs
directory containing the necessary .jar files can be downloaded as a zip archive (symmetric-android-VERSION.zip) from the SymmetricDS downloads
page. Alternatively, the libs directory can be generated by cloning the symmetric-ds GitHub repository and running the following command in the
symmetric-assemble directory:
[source, cli]
----
./gradlew androidDistZip
----
The first step to using SymmetricDS in an Android application is to unzip the jar files into a location where the project will recognize them.
The latest Android SDK requires that these jar files be put into a libs directory under the app directory of the Android application project.
[IMPORTANT]
In order to sync properly, the Sync URL of the corp-000 node must be updated to use the IP address of host rather than localhost.
Then, update the String REGISTRATION_URL in the DbProvider class of the Android project to the new Sync URL of the corp-000 node.
Next, set up an Android Emulator or connect and Android device. This can be done by opening the Android Virtual Device Manager. Click New and follow the steps. The higher
the Emulator's API, the better.
Run your Android Application by pressing the Run button in Android Studio. When prompted, select the emulator you just created. Monitor the
Console in Android Studio. Let the apk install on the emulator. Now watch the LogCat and wait as it attempts to register with your SymmetricDS
Master Node.
The core functionality of SymmetricDS on Android is implemented by starting the SymmetricService class as an Android service. This requires
building the SymmetricDS Android libraries using the steps mentioned above and adding them to your Android projects dependencies.
The SymmetricService Intent is defined in the AndroidManifest.xml using the following XML snippet:
[source, xml]
----
<service android:name="org.jumpmind.symmetric.android.SymmetricService" android:enabled="true" >
<intent-filter>
<action android:name="org.jumpmind.symmetric.android.SymmetricService" />
</intent-filter>
</service>
----
The SymmetricService Intent is started using the following java code:
[source, java]
----
Intent intent = new Intent(getContext(), SymmetricService.class);
// Replace extras with desired node configuration
intent.putExtra(SymmetricService.INTENTKEY_SQLITEOPENHELPER_REGISTRY_KEY, DATABASE_NAME);
intent.putExtra(SymmetricService.INTENTKEY_REGISTRATION_URL, REGISTRATION_URL);
intent.putExtra(SymmetricService.INTENTKEY_EXTERNAL_ID, NODE_ID);
intent.putExtra(SymmetricService.INTENTKEY_NODE_GROUP_ID, NODE_GROUP);
intent.putExtra(SymmetricService.INTENTKEY_START_IN_BACKGROUND, true);
Properties properties = new Properties();
// Put any additional SymmetricDS parameters into properties
intent.putExtra(SymmetricService.INTENTKEY_PROPERTIES, properties);
getContext().startService(intent);
----
=== Embedding in C/C++
A minimal implementation of the SymmetricDS client is written in C, which includes a shared library named "libsymclient" and a command line executable
named "sym" for synchronizing a database. It currently only supports the SQLite database. The SymmetricDS C library and client are built
from the following projects:
symmetric-client-clib:: This project contains most of the code and builds the libsymclient C library. It depends on libcurl, libsqlite3, and libcsv.
symmetric-client-clib-test:: This project links against the C library to runs unit tests. It also depends on the CUnit library.
symmetric-client-native:: This project links against the C library to build the sym executable.
The binaries are built using Eclipse CDT (C/C++ Development Tooling), which is an Integrated Developer Environment based on the Eclipse
platform. A distribution of Eclipse CDT can be downloaded or an existing Eclipse installation can be updated to include the CDT.
(See https://eclipse.org/cdt/ for information and downloads.) In the future, the projects above will switch to a general build system
like Autotools for automating builds, but for now Eclipse is required.
The "sym" executable can be run from the command line and expects the "libsymclient.so" library to be installed on the system.
If running from the project directories during development, the path to the library can be specified with the LD_LIBRARY_PATH environment
variable on Linux, the DYLD_LIBRARY_PATH on Mac OS X, or PATH on Windows. The executable will look for a "symmetric.properties" file
containing startup parameters in the user's home directory or in the current directory:
[source, cli]
----
LD_LIBRARY_PATH=../../symmetric-client-clib/Debug ./sym
----
It will also accept an argument of the path and filename of the properties file to use:
[source, cli]
----
LD_LIBRARY_PATH=../../symmetric-client-clib/Debug ./sym /path/to/client.properties
----
The client uses <<Startup Parameters>> to connect to a database, identify itself, and register with a server to request synchronization.
Here is an example client.properties file:
[source, text]
----
db.url=sqlite:file:test.db
group.id=store
external.id=003
registration.url=http://localhost:31415/sync/corp-000
----
The symmetric-client-native project is an example of how to use the SymEngine API provided by the C library. The C library
uses an object-oriented pattern and follows the same naming conventions as the Java project.
All symbol names in the C library are prefixed with "Sym". Each Java class is represented in C with a struct that contains member data
and pointers to member functions. Here is an example C program that runs the SymmetricDS engine:
[source, c]
----
#include "libsymclient.h"
int main(int argCount, char **argValues) {
// Startup and runtime parameters
SymProperties *prop = SymProperties_new(NULL);
prop->put(prop, SYM_PARAMETER_DB_URL, "sqlite:file:data.db");
prop->put(prop, SYM_PARAMETER_GROUP_ID, "store");
prop->put(prop, SYM_PARAMETER_EXTERNAL_ID, "003");
prop->put(prop, SYM_PARAMETER_REGISTRATION_URL, "http://localhost:31415/sync/corp-000");
// Uncomment to read parameters from a file instead
//SymProperties *prop = SymProperties_newWithFile(NULL, fileName);
SymEngine *engine = SymEngine_new(NULL, prop);
// Connects to database, creates config/runtime tables and triggers
engine->start(engine);
// Pull changes from remote nodes
engine->pull(engine);
// Create batches of captured changes
engine->route(engine);
// Push changes to remote nodes
engine->push(engine);
// Create a heartbeat batch with current host information
engine->heartbeat(engine, 0);
// Purge old batch data that has successfully synced
engine->purge(engine);
// Clean up
engine->stop(engine);
engine->destroy(engine);
prop->destroy(prop);
return 0;
}
----