/
TODO.txt
96 lines (83 loc) · 4.14 KB
/
TODO.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
TODO = *
DONE = +
* Go through every class and fix javadoc, remove unused methods, follow conventions, and fix generics/warnings
* Setup build for
+ 2.x
* 3.x
* Port CsvLoader and DataLoaderTest over. Check-in and make sure all unit tests work.
+ Clean up statistics. Take a pass through each of the reader/writers and make sure stats are recorded
+ Delete IDataLoaderFilter, IBatchListener
+ Introduce handler for conflict resolution. Implement default handler to start with
+ Setting for whether to update all columns or only the columns that changed
+ Plug in transform database writer
* Add settings to channel
* Lookup/choose datawriter based on channel settings
* clean up error handling in loader service
* SqlScript for Informix
* Implement a jdbc batch loader
* Use data processor for extract
+ Add dependency on spring-jdbc from symmetric-db (lob handlers and statement creators)
+ lob handling for extract
+ Finish DataExtractorService data selector
+ map target tables from routers
+ integrate into data extractor service
+ add transform writer
+ add staging file management api
* verify stats capturing, outgoing batch updates
* extract millis
* byte count
* add database reader filter
* unit test
+ purge job for staging manager
+ make sure memory buffer is cleared when state goes to done
* Make SymmetricDS case sensitive
* Get rid of delimited identifier "force" properties
* Add property to platform that identifies the default case used by the database
* Pre-process SQL to make sure symmetric tables are quoted and use the default case of the database
* Pre-process symmetric tables to default them to the case of the database
* Hook up JMX
+ Use data processor for streaming to file
+ Clean up logging
+ Use SLF4J and MDC (for capturing the engine & node name)
+ What to do with SqlTemplate? Move sqltemplate into dbdialect
+ Clean up symmetric.messages
* Can BatchInfo go away?
* Use HttpClient?
* Add table copy utility
* Clean up service tests
* Purge OK records instead of purging them at a later date
* Test batch inserts for router service
* Remove all references to data_ref
* Change the default server mode to multiserver
* SqlMap create constants for columns and table names
* SqlMap, TriggerTemplate format better
* Test timezone columns. Do other databases other than oracle and postgres have timezone columns?
Performance Improvement Opportunities
* Pluggable data loaders
* Make push/pull/routing of channels asynchronous
* Sync based on updated column values (timestamp or flag)
* Transaction Log Parsing (vs. triggers)
* Optimize triggers (don't capture old data. option to capture only primary keys)
* Routing optimizations - allow a channel to always sync to everybody. Update sym_outgoing_batch to have a PK of batch_id, node_id.
* Web Sockets protocol
* HttpClient (pool connections)
Documentation
* Change getting started to instruct user to update properties in the engines directory (for multiserver mode)
* Update documentation for extension points
3.0 Change Notes:
* The db.metadata.ignore.case property is no long available. All configured database names will be case sensitive
* db.spring.bean.name is no longer available
* dataloader will now load each batch as it arrives.
* datareload.batch.insert.transactional is no longer available
* routing.data.reader.type is no longer available. data_ref has been removed.
* jdbc datasources are no longer supported
* All data loader properties will be configurable by channel
* IDatabaseWriterFilter replaces IDataLoaderFilter
* IDatabaseWriterFilter replaces IBatchListener
* db.default.schema is no longer used
* No longer stop purge from running if there wasn't an initial load
* Extensions no longer have services injected into them. If they need acccess to services, they should implement ISymmetricEngineAware
* Node concurrency manager no longer allows the same node to make a second request while it already has a reservation
* Added stage management job (that purges staged files)
* Removed support for db.force.delimited.identifier.mode.on & db.force.delimited.identifier.mode.off
* Removed support for db.metadata.ignore.case