-
Notifications
You must be signed in to change notification settings - Fork 82
Program: Transaction Log Converter
Hiroki Kanezashi edited this page Oct 13, 2021
·
6 revisions
This program converts transaction simulator log files into CSV files for other post-processing applications such as GPML.
Usage:
python scripts/convert_logs.py [ConfJSON]
Arguments:
-
ConfJSON
: Configuration JSON file (e.g. conf.json)
Example:
python scripts/convert_logs.py conf.json
The output data schema (CSV columns) is defined at paramFiles/schema.json. The JSON file location must be specified at the configuration file like this:
{
"account": [
{"name": "acct_id", "valueType": "string", "dataType": "account_id"},
{"name": "dsply_nm", "valueType": "string", "dataType": "account_name"},
{"name": "type", "valueType": "string", "dataType": "account_type", "defaultValue": "SAV"},
//...
{"name": "lat", "valueType": "float"}
],
"transaction": [
{"name": "tran_id", "valueType": "string", "dataType": "transaction_id"},
//...
{"name": "alert_id", "valueType": "string", "dataType": "alert_id"}
],
"alert_tx": [
{"name": "alert_id", "valueType": "string", "dataType": "alert_id"},
{"name": "alert_type", "valueType": "string", "dataType": "alert_type"},
//...
{"name": "tran_timestamp", "valueType": "date", "dataType": "timestamp"}
],
"alert_member": [
{"name": "alert_id", "dataType": "alert_id"},
//...
{"name": "end", "dataType": "date", "defaultValue": 1000000},
{"name": "schedule_id", "dataType": "schedule_id"}
],
"party_individual": [
{"name": "birthPlaceCountry", "valueType": "string", "defaultValue": "US"},
//...
{"name": "isIndividual"}
],
"party_organization": [
{"name": "birthPlaceCountry", "valueType": "string", "defaultValue": "US"},
//...
{"name": "isIndividual", "valueType": "boolean", "defaultValue": false}
],
"account_mapping": [
{"name": "cust_acct_mapping_id", "valueType": "int", "dataType": "mapping_id"},
//...
{"name": "data_dump_dt", "valueType": "date", "defaultValue": 1}
],
"resolved_entities": [
{"name": "entityrefid", "valueType": "int", "dataType": "ref_id"},
//...
{"name": "reason", "defaultValue": "same"}
]
}
This JSON data contains eight lists with key names, and each list describes the order and types of attributes for each CSV file.