Skip to content

Latest commit

 

History

History
74 lines (54 loc) · 2.94 KB

README.md

File metadata and controls

74 lines (54 loc) · 2.94 KB
Raw

ETL Utility to Transform Data from One source to another

The utility can transform flat, hierarchical (table-like) data into flat, hierarchical data. It has adapters to read and write the data.

Type Factory Input Adapter Output Adapter Field Adapter for Reading Field Adapter for Writing Comments
POJO EtlPojoAdapterFactory createPojoInputAdapter createPojoOutputAdapter createPojoFieldReader createPojoFieldWriter
DB EtlDbAdapterFactory createDbInputAdapter createDbOutputAdapter createDbFieldReader createDbFieldWriter
SwiftMT EtlSwiftMtAdapterFactory createSwiftMtInputAdapter createSwiftMtOutputAdapter (TODO) createSwiftMtFieldReader createSwiftMtFieldWriter (TODO)

Configuration

XML

The conversion can be configured in XML. For this there should be two XML files: one - to define model structure, another - to set field-to-field mappings. For example:

db adapter schema.xml

db adapter conversions.xml

It can be completed using EtlAdapterConfigBuilder class. For example:

        Map<String, EtlModelAdapter> adapters = new EtlAdapterConfigBuilder()
                .readerStrategy(EtlPojoAdapterFactory.createPojoFieldReader())
                .writerStrategy(EtlPojoAdapterFactory.createPojoFieldWriter())
                .modelConfig(EtlAdapterConfigBuilderTest.class.getResourceAsStream("/schema.xml"))
                .conversionConfig(EtlAdapterConfigBuilderTest.class.getResourceAsStream("/conversions.xml"))
                .buildMap();

        EtlModelAdapter modelAdapter = adapters.get("conversion1");

        ...

        // source and target are objects of Map<String,Object> class

        modelAdapter.adapt(
                EtlPojoAdapterFactory.createPojoInputAdapter(source), 
                EtlPojoAdapterFactory.createPojoOutputAdapter(target)
        );

Please, see EtlAdapterConfigBuilderTest test case for details.

Java

Java conversion can be fulfilled using EtlAdapterBuilder. For example:

        EtlModelAdapter adapter = new EtlAdapterBuilder()
                .readerStrategy(readerStrategy)
                .writerStrategy(writerStrategy)
                .addFieldConversion("field1", "int", "field1_2")
                .addFieldConversion("field2", "string", "field2_2", "string")
                .buildAdapter();

        ...

        adapter.adapt(source, target);

Please, see EtlAdapterBuilder test for details.

Conversion rules

Currently Supported Types

Type Supported in Comments
String all built-in adapters
int all built-in adapters
double all built-in adapters

Types can be extend/customized via custom EtlFieldReaderStrategy/EtlFieldWriterStrategy