A Python application is aimed to convert mainframe EBCDIC data into Unicode ASCII delimited text files.
The converter consists of two parts
- Engine
- Engine conversion rules
Conversion rules are a driver to parse EBCDIC data.
-
Supported layouts
- Single schema
- Multi-schema fixed record length
- Multi-schema variable record length
-
Fixing anomalies in EBCDIC files
- Skip header
- Skip footer
- Remove invalid characters
-
Adding relationship keys
- parent-child
- parent-child-grandchild
-
Applicable encodings
- Python
- Java
Detailed information about conversion rules setup file can be found in Engine Rules manual. COBOL Engine Rules Dictionary manual describes how to convert COBOL data types into engine rules layout.
{
"description": "conversion rules sample",
"header": [
{
"name": "header", "type": "skip", "size": 100
}
],
"footer": [
{
"name": "footer", "type": "skip", "size": 52
}
],
"layouts": [
{
"layouttype": "main",
"layout": [
{
"name": "employee_name", "type": "string", "size": 55
},
{
"name": "wages", "type": "packedDecimal", "size": 8
},
{
"name": "employee_id", "type": "integer", "size": 4
}
]
}
]
}
It doesn't request any special installation if you are planning to use only Python encodings.
In case of using Java encodings, it has to be installed javabridge Python library. There is a constant in the code for including the javabridge library.
usage: ebcdic_parser.py [-h]
--inputfile "input file path"
--outputfolder "output folder"
--layoutfile "layout file"
[--outputdelimiter [delimiter]]
[--outputfileextension [extension]]
[--ignoreconversionerrors [yes/no]]
[--logfolder [log folder]]
[--pythonencoding [yes/no]]
[--encodingname [encoding name]]
[--grouprecords [yes/no]]
[--grouprecordslevel2 [yes/no]]
[--verbose [yes/no]]
Convert mainframe EBCDIC data to delimited text format. Version 2.1.0
Supported file formats:
(1) Single schema
(2) Multi-schema fixed length
(3) Multi-schema variable length
Features
1. Python doesn't include enough code pages, so it's added Java code pages as well.
Java is implemented in javabridge module. If javabridge module is not installed,
Java functionlaity can be disabled changing JAVABRIDGEINCLUDED = True to False in Python code
optional arguments:
-h, --help - Show this help message and exit
--inputfile "input file path" - Input EBCDIC file path
--outputfolder "output folder" - Output folder to store delimited files
--layoutfile "layout file" - Layout file path
--outputdelimiter [delimiter] - Output text file delimiter
--outputfileextension [extension] - Output text file extension
--ignoreconversionerrors [yes/no] - Ignore any conversion error
--logfolder [log folder] - Output folder to store log file
--pythonencoding [yes/no] - Use Python encoding rather than Java
--encodingname [encoding name] - Code page name to encode characters (Python or Java)
--grouprecords [yes/no] - Create relationships between records
--grouprecordslevel2 [yes/no] - Create relationships between records for level 2
--verbose [yes/no] - Show information on screen
Exit codes: 0 - successful completion, 1 - completion with any error
Those tests cover testing of each data type.
test_functional.py
The tests are needed to test entire functionality for each feature.
- Tool to create samples
converter_to_ebcdic.py
- Run tests
test_system_311_calls_for_service_requests_all_strings.py
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
This project is licensed under the MIT License - see the LICENSE file for details
- Thank you, javabridge team, for development and prompt support of the library
- https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_72/rzasd/dtdf.htm
- http://www.mainframestechhelp.com/tutorials/cobol/
- http://www.fileformat.info/
- City Toronto Open Data
- Reading COBOL Layouts