Skip to content

ironwork 0.5.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Oct 07:57
· 192 commits to main since this release
v0.5.0
3d122ba

Summary

ironwork 0.5.0 makes run --vm and cics --vm write the same evidence journal and coverage report as the interpreter, and does the same for a compiled module, so cobolwork can verify a run of either executor. It also reads GnuCOBOL and Micro Focus extensions on request, runs DBCS data, and gives GnuCOBOL's results where a migration needs them.

  • VM evidence: run --vm and cics --vm take --evidence, its traces and --coverage. On cobolwork's 438 test fixtures, the site's programs and CardDemo's 31 programs the two executors gave the same journals, coverage, output and files, and cobolwork's verifier accepts them.
  • Modules: ironwork cics x.iwm runs a module's first program as a task's first program, and --coverage and --evidence work on a module run. A module run gives the journal and coverage report its source run gives.
  • Code as text: ironwork dump prints a module's generated code as readable text, with data names, subscripts and source lines, in place of Rust debug output.
  • Dialect: --dialect gnucobol gives cobc's results where ironwork's assumptions chose otherwise: intermediate arithmetic, DISPLAY of signed and binary items and of numeric literals, ACCEPT at the end of SYSIN, dynamic CALL of an ENTRY name, a shorter EXTERNAL record, and NOINVDATA comparisons. --dialect ibm is the default and changes nothing.
  • Compliance: --compliance extended reads six extensions found in real programs, each with a warning: free-form source, level-78 and CONSTANT entries, <>, & between literals, BINARY-SHORT, -LONG and -DOUBLE, and a PROGRAM-ID with no IDENTIFICATION DIVISION. strict is the default and changes nothing.
  • DBCS: PICTURE G and N under NSYMBOL(DBCS), USAGE DISPLAY-1 and GRAPHIC host variables compile and run under all eleven mixed CCSIDs of the Programming Guide's Table 47, on both executors.
  • VM speed: the VM meets each speed target set for it: table search runs in 0.187 of the interpreter's time and call-heavy code in 0.184 (target 0.20), packed decimal in 0.294 (target 0.33), and sequential I/O in 0.66 of cobc -O2's time (target 1.25).
  • Differential testing: CI runs all 384 NIST CCVS85 programs the interpreter runs on the VM too and compares them. ironwork fuzz --differential runs generated inputs on both executors and keeps a smaller input for each way they differ.
  • Compiler checks: ten shapes that Enterprise COBOL refuses are now refused when compiling, not when generating code, and five shapes it accepts now run on the VM.
  • Subprogram fuzzing: ironwork fuzz --interface runs a subprogram as a caller would, and run --argument gives a program its USING items.
  • IBM's rules: twelve results that departed from Enterprise COBOL's documented rules now follow them, among them a quotient's decimal places, FUNCTION MOD's sign, floating-point exponentiation and DISPLAY of national data; seven more NIST CCVS85 programs run clean (376 of 384).
  • Fixes: INITIALIZE of a reference-modified item no longer writes past the item, and the evidence ledger lock is broken by cobolwork's rule.

Limit: the VM is still opt-in. run and cics use the interpreter unless given --vm, --vm with --serve is refused, and a run that reaches something the VM does not run yet stops with exit status 243.

Added

  • run --vm and cics --vm write evidence and coverage. --evidence, --trace-input, --trace-statements, --trace-marker and --coverage are accepted with --vm.

  • ironwork cics x.iwm runs a module under CICS on the VM with every cics flag except --serve and --serve-public, which are refused by name. Under --screens a transaction's program is found as a CALL of it finds one, in the module first. The compile flags and --provenance are still refused on a module run.

  • --coverage and --evidence on a module run. Coverage outlines the programs of the first program's source from the module's paragraphs and debug table. A module's DEBUG record now holds the SHA-256 and length of each source file and COPY member its compile read, so the journal names them as a source run does.

  • ironwork dump prints each program's ranges and blocks as text, one op or terminator to a line. docs/lir.md section 13 gives the form.

  • --dialect ibm|gnucobol on run, check, cics, job, compile and fuzz. The dialect is recorded in the provenance statement, the evidence journal's argv and the module's options.

  • --compliance strict|extended as a compile option. Each extension use is a warning, IWX0001-W to IWX0006-W. docs/compliance.md gives each meaning and what stays refused. A module's run reads a source it CALLs under the compliance level of the module's first program.

  • DBCS: PICTURE G and N under NSYMBOL(DBCS), B for a DBCS space, USAGE DISPLAY-1, G and N literals, and the eleven mixed CCSIDs through CODEPAGE. MOVE, comparison, INITIALIZE, reference modification, the DBCS and KANJI class tests, STRING, UNSTRING, INSPECT, LENGTH, NATIONAL-OF, JSON and XML GENERATE, and GRAPHIC and VARGRAPHIC host variables follow IBM's rules. Under a mixed code page an alphanumeric item may hold DBCS characters between shift-out and shift-in. The four Japanese CICS programs in the test corpus compile. Assumptions C282 to C286 record what the manuals leave open.

  • ironwork fuzz --differential runs a batch program twice per input, through run and run --vm, under one statement limit (--hang-limit, or 1,000,000). A pair that differs in exit status, abend, standard output, standard error or a DD's data set is kept as divergence-N/ with the command that repeats it. The exit status is 1 when any input differs.

  • ironwork fuzz --interface and ironwork run --argument PATH|OMITTED. A subprogram is run with one argument per USING item built from its LINKAGE record, and where a program in its directory or an -L library CALLs it by name, a run follows that CALL's shape. Programs with a pointer argument, IMS programs and CICS programs are refused. The manifest's format is ironwork-fuzz-interface/v1, which a reader of ironwork-fuzz/v1 refuses.

  • The JCL runner's DFSORT and IDCAMS breadth, described in 0.6.0, is in this release's code too.

  • The VM dispatches faster: integer arithmetic takes a fast path where the plan's inner and last decimal places allow it, and the benchmark table in docs/benchmarks.md gives each class's time against the interpreter and cobc -O2 (release build, five interleaved runs).

Changed

  • The load-module format is 0.5. It went from 0.3 to 0.4 when Options gained the dialect and an arithmetic plan its second dmax, and to 0.5 when the DEBUG record gained the source files. A module of an earlier format is refused with "compile the source again".
  • A module's bytes depend on the order of the -I libraries, which numbers the library each source file is recorded under.
  • A journal names a file from the innermost root that holds it. A COPY member in lib/ beside a program was lib/UPCASE.cbl under the program's root and is now UPCASE.cbl under the library. This covers input, call, sink, statement and abend records, a fuzz manifest's abend file, and the source in a job's coverage report.
  • The compiler refuses, with a severe message, ALL with a numeric literal or another ALL; a condition-name used as data; an arithmetic expression or numeric function compared with a non-numeric operand; MAX, MIN, ORD-MAX and ORD-MIN arguments of different classes or a pointer; a figurative constant as a function argument outside an expression; ALL subscripts in a function of fixed arity; PERFORM VARYING of a non-numeric item or FROM or BY an arithmetic expression; SEARCH VARYING an item that is neither an index nor an integer; PROCEDURE DIVISION RETURNING an item that is not an 01 or 77 LINKAGE item; and an EXEC CICS HANDLE CONDITION, AID or ABEND label that names no paragraph or section. No CCVS85 program is newly refused.
  • STRING and UNSTRING count national and DBCS character positions, as the Language Reference says.
  • A function prototype is no longer outlined in a source run's coverage report; a module holds none, and the function it declares is reported as called.
  • A bad --dialect value exits 246, the usage status, on run, cics and job.

Fixed

  • INITIALIZE of a reference-modified item treats it as one alphanumeric receiver, as the Language Reference does. N (2:2) of a PIC 9(4) item wrote four bytes from N's second, past the end of storage when N was the last WORKING-STORAGE item, and both executors panicked. X (1:2) of a PIC X(6) item blanked all six bytes (C300).

  • A callee that sets the address of its RETURNING item to NULL gives the interpreter's S0C4 on the VM, naming the item at the CALL, INVOKE or function reference. The VM stopped the CALL or INVOKE before.

  • A cics task the VM stopped, or whose program code generation refused, now closes its journal and reaches the ledger. It returned without a close record.

  • The evidence ledger lock is broken by cobolwork's rule: the holder is checked, the age is read from the lock, and a stale file is renamed aside before removal. ironwork broke any lock older than a minute by file time, and two writers that found one stale lock could both remove it and fork the ledger. A lock that names no pid is recorded with a null holder.

  • FILE STATUS is marked as input only by a verb that reads the key. The interpreter's OPEN of an indexed file marked it and the VM's did not, and a status that once held input passed it on to every later status. A random READ, START or DELETE marks the key it reads on both executors.

  • A COMP-1 or COMP-2 PERFORM VARYING variable steps in floating point. The interpreter added BY in fixed point and dropped the fraction of a step such as 1.5.

  • RANGE with a floating-point argument is floating point on both executors. ALL with a national literal, and JSON GENERATE, XML GENERATE and JSON PARSE of an item with P scaling, now run on the VM.

  • A quotient carries the dividend's decimal places less the divisor's, or dmax, whichever is greater (Programming Guide p. 795).

  • FUNCTION MOD takes the divisor's sign (Language Reference p. 601).

  • A LINKAGE 01 that REDEFINES another gets that record's address (p. 225), and a group's SIGN clause reaches the signed zoned items under it (p. 231).

  • INSPECT examines a signed zoned item as its unsigned digits (p. 359), and SEARCH VARYING the table's own index steps that index (p. 437).

  • An ALL literal is cut to the length of the item it is compared with (p. 17).

  • Floating-point exponentiation and decimal exponents are evaluated as IBM's precision rules give them (Programming Guide pp. 796, 800).

  • DISPLAY writes national data unconverted except UPON CONSOLE (Language Reference p. 333).

  • ironwork fuzz keeps statement coverage for every run it makes.

Upgrading

  • Recompile every load module from 0.4.1 or earlier once. They are format 0.3 or earlier and this release reads 0.5; ironwork run x.iwm refuses them with "compile the source again", and ironwork compile on the source writes a new one.
  • From this release, a load module stays readable across 0.x releases where the format allows. A reader reads every format from the oldest it can still decode (0.5) up to its own, and a newer format by skipping the optional sections it does not know; it refuses an unknown required section. A change that cannot be made additive moves the oldest readable format and refuses older modules, as 0.5 does for 0.4 and earlier. 1.0 freezes the format. A module beside newer source still runs: the loader does not compare file times.
  • Journals and fuzz manifests that name a file under a library now give the path relative to the innermost root. A tool that matched the old lib/NAME.cbl form needs the new one.
  • A program the compiler now refuses for one of the ten shapes above is reported at compile time. Programs that reached a code-generation refusal (exit 242) with one of them now exit 241.
  • Programs that STRING or UNSTRING national or DBCS data now count characters, not bytes.
  • The compiler now refuses what Enterprise COBOL refuses: DISPLAY of an integer or numeric intrinsic function (MAX and MIN of numeric arguments included), DISPLAY UPON a name that is not an output device (for example UPON CRT), and a VALUE literal that is not numeric on a numeric item. Such programs now stop at compile time with exit 241.

Install

ironwork for COBOL 0.5.0. Builds for Linux (static), macOS and Windows, and the npm package @portll/ironwork. The ironwork-tls archives are the same builds with TLS for --sql-db (rustls, sslmode=verify-full). The Python package is on PyPI as ironwork, and the crates are on crates.io. Each file carries SLSA build provenance: gh attestation verify -R Portll/ironwork.

Assets 14
Loading