CTRAN

An attempt to reverse-engineer and recreate CTRAN.EXE, the Psion Object Oriented C preprocessor. In Object Pascal.

Aims

• A drop-in replacement for the original CTRAN.EXE in the Psion SIBO C SDK, replicating both the input switches and the generated files.

• Run on modern OSes and DOS.

• Replicate files generated by Psion CTRAN 5.02.

• Allow multiple include sources.

Targets for v1.0.0

• ✓ Psion Category file lexer

• ✓ Psion Category file parser/AST builder

• ✓ Command line parser [basic, but works well enough!]

• ✓ Build class dependency tree

• ✓ Generate .ext files

• ✓ Generate .c files

• ✓ Generate .g files

• ✓ Handle sub-category (.CL) files

• ✓ Generate files designed to only work with the SDK

• ✓ Generate skeleton method files

• ✓ Generate .asm (TASM assembly) files

• ✓ Generate .ing (TASM assembly header) files [PROPERTY and TYPES generation is basic — see below]

• ✓ Generate .lis (class listing) files

What is (or was) CTRAN?

CTRAN.EXE is a pre-processor for the object-oriented part of Psion’s SIBO C SDK, targeting the SIBO/EPOC16 range of portable devices. It takes a .cat category file containing class definitions (methods, properties, etc) and generates the following files:

• A .c file

• A .g header file, to be `#include`d with the class’s methods.

• A .ext file with a summary of the class that can be reused by CTRAN

• A .asm file

• A .ing file (assembly header)

• A .lis file

In regular software development, only the first three are generated.

The original CTRAN.EXE can also generate alternative versions of files for use only with the SDK, plus skeleton source files on which methods can be built.

The problem

My main gripe with the original CTRAN.EXE is that it can only include files from one folder. So, if you want to include the ROOT class in OLIB (the main OO library) and one of your own classes, they both have to be in the same location. This means that you have to store every generated file inside the SDK folder.

I did not like this.

Examples

Generate .C and .G files from VECTOR.CAT on DOS and put them in the current directory.

ctran vector.cat -ec:\sibosdk\include -c -g

Generate .C and .G files from LCDTEST.CAT on Linux, where the SDK is in /home/user/sibosdk, but put the resulting files in /home/user/myproject/classes/:

ctran lcdtest.cat -e/home/user/sibosdk -c/home/user/myproject/classes -g/home/user/myproject/classes

Generate .C, .G and .EXT files from WARI.CAT on DOS that can only be used with the SDK. (Note that specifying the .CAT extension is optional)

ctran wari -ec:\sibosdk\include -c -g -x -s

An improvement over the original CTRAN.EXE is that you can specify a default output path using the -p switch. You can also override -p with individual switches for each file type you are generating. So, if you want to generate .C, .G and .LIS files for DEMO.CAT, you want the default output path to be demo/, but you want the .LIS file to go to the current working directory:

ctran demo -ec:\sibosdk\include -pdemo/ -c -g -l.

State of the project

What the CTRAN-ng can currently do

CTRAN-ng has feature-parity with Psion CTRAN 5.02, minus the OVAL and (broken) EPOC32 featues. Most people only need the ability to generate .C and .G files, which it does well. It can be used as a drop-in replacement for the original Psion CTRAN.EXE. (In fact, it fixes some minor issues with Psion CTRAN, and generates cleaner, more readable files.)

In addition, it can act as a Psion SIBO category file validation tool, for use with category (.CAT), sub-category (.CL) and external category (.EXT) files. It extends the original -v (verbose) switch by adding and combining the following options:

• l Show lexer debug output

• t Show token table

• p Show parser debug output

• a Show abstract syntax tables

• r Reconstruct all input files using the abstract syntax tables

For example, the following DOS command line will take the DEMO.CAT file and show the token table and abstract syntax tables:

ctran demo -ec:\sibosdk\include -vta

Finally, the -e switch can take multiple paths, separated by a semicolon. So, if you want to include external (.EXT) files from the SDK and a separate project location, you can use:

ctran demo -ec:\sibosdk\include;p:\projects\myproject -c -g -x

Or, if the include files are in the current working directory:

ctran demo -ec:\sibosdk\include;. -c -g -x

This also works with *NIX paths, if you add speech marks.

ctran demo -e/home/user/sibosdk/include;/home/user/myproject/inc -c -g -x

What it can’t do (or is currently broken/weird)

"Shadow" classes

There is a small issue when processing classes that only have `DEFER`red methods (something I’ve called "shadow" classes). The original way of handling this is to completely skip these classes, which CTRAN-ng doesn’t do. This shouldn’t impact existing code, but it will be fixed in the future.

Borland Turbo Assembler file generation

TASM file generation does work and matches the original Psion CTRAN. However, I do not believe that it is reliable.

The method used to convert the C in category file PROPERTY and TYPES sections to 8086 assembly is rudimentary at best. There is barely any error checking and some lines in PROPERTY and TYPES are completely ignored. Like I said, this does replicate Psion `CTRAN’s behaviour, but I don’t believe that means that it generates working assembly.

It is highly recommended that anyone using these generated files checks them thoroughly before using them.

The order that some lines are generated

Some lines at the start of files are geneated in a different order to Psion CTRAN. I haven’t been able to work out how the original CTRAN is generating these lines in the way that it does, although I am sure it has something to do with the way that it is storing classes. However, as these are just declarations of macros and the only thing different is the order, it has no affect on how TopSpeed C compiles them.

Psion OVAL

It can’t generate files for use with Psion OVAL (see below).

Different versions of Psion’s CTRAN.EXE

There are two commonly available versions of Psion CTRAN:

• 3.43: This came with the SIBO C SDK 2.20 and is (probably) the version that most people using the SDK would have.

• 5.02: This came with Psion OVAL 1.21, an IDE based on Visual Basic 6 that could be used to create new controls for use in EPOC16 applications, as well as entire EPOC16 applications. This version of CTRAN.EXE generates cleaner code than the previous version, and fixes some (but not all) bugs in 3.43. It adds the ability to generate files for OVAL controls. It also adds a (broken) option for generating code for EPOC32.

The aim of CTRAN-ng 1.0.0 is to mimic the code that CTRAN 5.02 generates. OVAL-related features won’t be in 1.0.0, but might be added in a later release. As CTRAN-ng is only designed to be used for making EPOC16 apps, the EPOC32 "feature" will not be added.

Why is this project written in (Object) Pascal?

I want this to run on all modern operating systems, as well as DOS. I didn’t want to write it in C or C++, so I went hunting for another language/compiler.

The only compiler that I could find that was still maintained and stable was Free Pascal. It targets pretty much everything, including Linux, Windows, macOS, Haiku, *BSD…​ and both 16-bit and 32-bit DOS.

Luckily, Object Pascal is just high-level enough, taking away some of the worries of memory management and how to handle dictionaries/trees, while letting me go low-level when I want to.

Unfortunately, some of Free Pascal’s units (e.g. classes, generics.collections) are too big to fit into 64 KB code blocks. Because of this, CTRAN-ng for DOS is a 32-bit executable using a CWSDPMI.EXE, a copy of which is bundled with CTRAN-ng for DOS. As most people will be running the Psion SIBO C SDK in DOSBox, it shouldn’t affect anyone using CTRAN-ng directly, or if it is run from a TopSpeed project (.pr) file. However, it has been known to clash with Borland Make, as its 16-bit DPMI extender prevents CTRAN-ng from running. In such situations, it is recommended that you switch your project to using GNU Make from the DJGPP project. Alternatively, TopSpeed project files provide a good alternative without needing an external Make app.

The future

At some point the entire SDK needs to be moved away from DOSBox. This will require rewriting every single tool that was included in the SDK, plus the TopSpeed C compiler.

There is a lot of work to do, but recreating CTRAN as a FOSS project is the first step.