Program: Transaction Log Converter

Transaction Log Converter

This program converts transaction simulator log files into CSV files for other post-processing applications such as GPML.

Usage:

python scripts/convert_logs.py [ConfJSON]

Arguments:

• ConfJSON: Configuration JSON file (e.g. conf.json)

Example:

python scripts/convert_logs.py conf.json

The output data schema (CSV columns) is defined at paramFiles/schema.json. The JSON file location must be specified at the configuration file like this:

{
  "account": [
    {"name": "acct_id", "valueType": "string", "dataType": "account_id"},
    {"name": "dsply_nm", "valueType": "string", "dataType": "account_name"}, 
    {"name": "type", "valueType": "string", "dataType": "account_type", "defaultValue": "SAV"},
    //...
    {"name": "lat", "valueType": "float"}
  ], 
  "transaction": [
    {"name": "tran_id", "valueType": "string", "dataType": "transaction_id"}, 
    //...
    {"name": "alert_id", "valueType": "string", "dataType": "alert_id"}
  ],
  "alert_tx": [
    {"name": "alert_id", "valueType": "string", "dataType": "alert_id"},
    {"name": "alert_type", "valueType": "string", "dataType": "alert_type"},
    //...
    {"name": "tran_timestamp", "valueType": "date", "dataType": "timestamp"}
  ],
  "alert_member": [
    {"name": "alert_id", "dataType": "alert_id"},
    //...
    {"name": "end", "dataType": "date", "defaultValue": 1000000},
    {"name": "schedule_id", "dataType": "schedule_id"}
  ],
  "party_individual": [
    {"name": "birthPlaceCountry", "valueType": "string", "defaultValue": "US"},
    //...
    {"name": "isIndividual"}
  ],
  "party_organization": [
    {"name": "birthPlaceCountry", "valueType": "string", "defaultValue": "US"},
    //...
    {"name": "isIndividual", "valueType": "boolean", "defaultValue": false}
  ],
  "account_mapping": [
    {"name": "cust_acct_mapping_id", "valueType": "int", "dataType": "mapping_id"},
    //...
    {"name": "data_dump_dt", "valueType": "date", "defaultValue": 1}
  ],
  "resolved_entities": [
    {"name": "entityrefid", "valueType": "int", "dataType": "ref_id"},
    //...
    {"name": "reason", "defaultValue": "same"}
  ]
}

This JSON data contains eight lists with key names, and each list describes the order and types of attributes for each CSV file.