Skip to content

ironwork 0.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 03 Oct 13:23
· 122 commits to main since this release
v0.4.0
fd3b620

Summary

ironwork can now compile a program to a load module and run it, and a second executor, the VM, runs what the interpreter runs. Every test program lowers (1482 of 1482), and the VM runs 680 of 682 test programs to the end with no difference from the interpreter; the 2 stops are FUNCTION RANDOM in a place and FUNCTION UUID4.

  • Load modules: ironwork compile writes a module, ironwork dump reads one, and ironwork run x.iwm runs its first program. A CALL, a function invocation, INVOKE, LINK and XCTL find their programs in the modules already read, then in NAME.iwm in the program libraries, then in source.
  • VM: ironwork run --vm and ironwork cics --vm now run file I/O, SORT and MERGE, the Report Writer, JSON and XML GENERATE and PARSE, EXEC CICS and EXEC SQL, INVOKE and the JNI services, LE callable services and the virtual printer, EXTERNAL and GLOBAL storage, and NUMCHECK and PARMCHECK programs.
  • User-defined functions: FUNCTION-ID definitions and prototypes compile, an invocation is checked against its definition, and the function runs as a CALLed program runs. IBM's docalc example gives the Programming Guide's result 010230.
  • CICS fidelity: HANDLE ABEND follows the rules for logical levels, across CALL, LINK and XCTL. STOP RUN below level one returns to the LINKing program, and each LINK, XCTL and HANDLE ABEND PROGRAM exit starts a run unit with its own programs, EXTERNAL data and heap.
  • Recursion: a CALL of an active program that is not RECURSIVE ends with U4038 and IGZ0064S, and a RECURSIVE main program can CALL itself.
  • Fuzzing: ironwork fuzz now fuzzes variable-length and relative files, a main program's PARM, a job's data sets, in-stream data and step PARMs (--job), and a CICS task's COMMAREA and terminal input (--cics). It keeps a loop an input caused, and runs each kept abend again at OPTIMIZE(2).
  • Compiler options: OPTIMIZE(0|1|2) is read, and at 1 and 2 a zoned item compared with zero is compared by its bytes, as IBM's example shows.
  • National data: INSPECT of a NATIONAL item counts, matches and replaces in characters, not bytes.
  • VSAMOPENFS: a VSAM data set left open for output, by a run that ended under TRAP(OFF), opens next time with status 97 under VSAMOPENFS(COMPAT) or 00 under SUCC.
  • Evidence: a run given --evidence and --trace-input records whether an input byte may be in a sink's operand, including through EXEC CICS.

Limit: the VM still stops at FUNCTION UUID4, FUNCTION RANDOM in a subscript, a SEND MAP or RECEIVE MAP whose MAP is held in a data item, and a CALL, LINK, function or method that does not lower; the interpreter stays the default, and ironwork cics --serve keeps it.

Added

  • Load modules run. ironwork run x.iwm runs a module's first program on the VM. Each program read from a module passes the LIR verifier first, since a module is untrusted input. A module beside newer source is the one that runs: the loader never compares file times. A user-defined function's directory entry holds its external name, the BMS section holds the mapsets a program names by a literal, and COMMON is recorded.
  • FUNCTION-ID. Definitions and prototypes parse to END FUNCTION. An invocation is checked for argument count and for each BY REFERENCE item against its parameter's description. The RETURNING item's value is the invocation's. Assumptions C270 to C274 record what the manuals leave open. A definition whose parameter or RETURNING record holds an OCCURS DEPENDING ON table does not lower.
  • VM coverage. File statements, SORT, MERGE, the Report Writer, JSON and XML statements, EXEC CICS (with a handler table per activation, HANDLE ABEND at each logical level, and LINK and XCTL as new activations), EXEC SQL, INVOKE, SELF, SUPER, NEW, the JNI services, LE callable services and the virtual printer all run on the VM through the same rt services the interpreter calls.
  • Lowering. The following now lower and run on the VM: items after an OCCURS DEPENDING ON table, SET TO ENTRY and a CALL through its pointer, INITIALIZE with FILLER, VALUE, REPLACING or DEFAULT, RECORD VARYING DEPENDING ON, a SEARCH whose OCCURS DEPENDING ON count is evaluated once, a literal the code page cannot encode (the VM abends where the interpreter does), a FILE STATUS that names no data item, EXTERNAL and GLOBAL storage, files and declaratives, user-defined function invocations, EXEC CICS HANDLE ABEND, and a literal MAP's symbolic map as the FROM or INTO of SEND MAP and RECEIVE MAP.
  • ironwork cics --vm runs each task, including a --screens conversation, on the VM. A program lowering refuses, or a run that reaches what the VM does not run, stops with the message run --vm gives and status 12.
  • OPTIMIZE(0|1|2) from a CBL or PROCESS card, and --optimize=N for the compiler invocation's level, which a card outranks. Under NOINVDATA at 1 or 2, an unsigned zoned integer compared with zero or with one of its own length is compared by its bytes; a numeric literal of value zero counts as ZERO (C262).
  • VSAMOPENFS takes effect. A VSAM (indexed or relative) data set keeps an open-for-output indicator, set by OPEN OUTPUT, I-O or EXTEND and cleared only by its successful CLOSE, as DFSMS keeps it. An OPEN that finds it on verifies the data set and succeeds with status 97 under VSAMOPENFS(COMPAT), the default, or 00 under SUCC (Programming Guide pp. 199, 424; Language Reference p. 303); a 97 runs the file's EXCEPTION/ERROR procedure but never ends the run. Under TRAP(ON), Language Environment's default, COBOL closes a run's files even when it abends, so only a program check or system abend under TRAP(OFF) leaves a data set open (C152, C220). The indicator is a file beside the data set, <path>.open-for-output; ironwork job deletes it with its data set, and DEFINE CLUSTER and REPRO clear it.
  • --statement-limit N on ironwork run and job. After N statements have started the run ends with S322 at the next one, as z/OS ends a step that runs past TIME=. Each job step gets N (C241).
  • ironwork run --parm TEXT gives the program the PARM an EXEC PGM= would, as Language Environment passes it. ironwork job takes --step-parm STEP=TEXT, --instream STEP.DD=path and --coverage FILE.
  • ironwork fuzz:
    • feeds files whose records have more than one length, each behind an RDW, and relative files with some slots empty; generates packed fields with a zero pad nibble and national fields as UTF-16;
    • varies a main program's PARM up to 100 characters when its one USING item is a halfword length group;
    • --job JCL fuzzes the data sets, in-stream data and step PARMs of a job; only an abend placed at a COBOL statement is kept;
    • --cics fuzzes a CICS task's COMMAREA and the operator's typing into the maps it RECEIVEs, with new home, tab and string keys in screen scripts;
    • keeps a timeout, up to three a run, as a loop the input caused when it ends in S322 under --statement-limit (--hang-limit), unless ACCEPT had found SYSIN at its end;
    • keeps an S806, up to five a run, only where the CALLed name is in the input and a marker name ends in S806 at the same CALL;
    • runs each kept abend again at --optimize=2 and records optimized in the manifest, true when the same abend came at the same place;
    • names the manifest's format ironwork-fuzz/v1 and ships docs/fuzz-manifest.schema.json;
    • refuses every flag it would not use (among them --declare, --sql-record, --expected, --proclib, --statement, --base and --head), which it used to ignore.
  • Input tracking. rt::taint keeps one bit per byte of run-unit memory. READ, ACCEPT from SYSIN, EXEC SQL rows, PARM and a CICS task's COMMAREA set it, and each write carries what its statement has read. Under EXEC CICS, RECEIVE, RECEIVE MAP, READQ TS and TD on a found item, and a file READ, READNEXT or READPREV on a found record mark input. ironwork cics takes --trace-input and --trace-statements. A sink that would read false reads null after SORT, the Report Writer, XML, JSON, OO or LE services, which are not followed yet.
  • compare and job equivalence statements (equivalence-v1, job-equivalence-v1) carry clock, the fixed time as ISO 8601 UTC.
  • The VM is timed beside the interpreter and cobc in the benchmarks, and the script fails if the VM's output differs from the interpreter's.

Changed

  • Under CICS, an INVOKE now abends IRONWORK before any operand is evaluated, because object-oriented COBOL cannot run under CICS and class definitions and methods cannot contain EXEC CICS (Programming Guide pp. 495, 679). A program with an INVOKE still compiles (C147).
  • STOP RUN below CICS logical level 1 returns to the LINKing program. At the task's first level it still ends the task. The Programming Guide's table says STOP RUN ends the whole transaction under CICS (p. 546); C144 records the conflict and the choice.
  • Each LINK, XCTL and HANDLE ABEND PROGRAM exit starts a fresh run unit. Every program starts in its initial state there with storage of its own, EXTERNAL records and files and the CEEGTST heap are the run unit's, and when it ends its open files are closed and the level above has its own back. A LINK or XCTL may now reach the task's first program, and a menu program can be XCTLed back to (C145, C126, C148).
  • A CALL of an active program that is not RECURSIVE ends with U4038 and IGZ0064S, naming the outermost program of its source. It ended with an ironwork abend before, and every CALL of the run unit's first program was refused (C127).
  • A CALLed program runs at its caller's logical level. A static CALL shares the level's HANDLE ABEND exit and HANDLE CONDITION table; a dynamic CALL suspends the caller's under CBLPSHPOP(ON) and restores them on return. RETURN or XCTL in a CALLed program ends the level. A label reached from another activation abends APC2 (C233 to C238).
  • A HANDLE ABEND label is entered as a GO TO from the HANDLE ABEND command, so PERFORMs the abend left stay armed. The level's exit and PUSH HANDLE stack survive XCTL (C236, C239, C146).
  • RETURN raises INVREQ with RESP2 2 for COMMAREA, IMMEDIATE or CHANNEL below the task's first level, and takes TRANSID at any level; it raised INVREQ with RESP2 0 for TRANSID too. A COMMAREA length outside 0 to 32763 raises LENGERR with RESP2 11 (C143, C128).
  • ironwork run exits 255 for a RETURN-CODE outside 0 to 255. It used to truncate to eight bits, so 256 exited 0 and -1 exited 255 by wrapping. 0 to 255 are unchanged, and the evidence journal records the true value.
  • ironwork cics --serve refuses an address that resolves anywhere but loopback unless --serve-public is given. The server asks for no credentials, and it now says so on standard error when it serves publicly.
  • ACCEPT from SYSIN reads each line as an 80-byte card, a shorter line padded with spaces, and fills the receiver from consecutive cards unconverted, as the Language Reference says (pp. 307-308). An empty card gives a numeric receiver spaces; a non-digit is a data exception where the item is next read as a number. Assumption C261.
  • A function argument expression takes part in the arithmetic of the expression that holds the function: in floating point where that is, otherwise with the larger dmax. FUNCTION LOG(1 / 10) no longer takes LOG of 0.
  • INSPECT of a NATIONAL item counts, matches and replaces in national characters, and a figurative constant is one national character. ironwork now refuses, when it compiles, an INSPECT operand whose usage differs from the inspected item's (Language Reference pp. 355-359). Assumptions C230 to C232.
  • A CALL's NOT ON EXCEPTION phrase runs after the CALL's nesting is released, as INVOKE and LE service CALLs already did, so the interpreter no longer abends at the nesting limit where the VM ran it.
  • An abend in a CALLed program names that program's own source file, and so does an abend in a cics --screens task whose program a -L library supplied. A job step's abend names its source by the rule run uses.
  • NUMCHECK, ZONECHECK and PARMCHECK programs lower (the 0.3.0 refusal is lifted), with ZON(LAX)'s tolerance and the tests the compiler removed carried on each place. A serial SEARCH or an ALL ZERO comparison that NUMCHECK may test twice still does not lower.

Fixed

  • OPEN ... NO REWIND without WITH is accepted, as the Language Reference's OPEN format 1 shows. It was read as a second file name and refused as "no file named NO".
  • ironwork cics: a key that started a task the task never RECEIVEd was handed to every later task, so COSGN00C ran forever on PF5. RECEIVE MAP of unformatted input now raises MAPFAIL, as CICS documents, instead of stopping the run.
  • JSON GENERATE tests a group for null as JSON NULL or INDICATING names it, before its members. A group whose members are all ignored is left out, not written as an empty object. JSON PARSE gives JSON-CODE 104 for a number bound for an alphabetic item.
  • A GLOBAL item inherited by a contained program is now found when JSON PARSE checks its PICTURE category.
  • An XML register read is counted for input tracking on the VM.
  • On the VM, a PERFORM inside the paragraph of a SORT stopped by RELEASE or RETURN under SORT-RETURN 16 gets no resume, as the interpreter has it.

Upgrading

  • Recompile every load module. This release reads load-module format 0.3. Every module compiled by ironwork 0.3.0 or earlier is refused with "load module format …; this ironwork reads 0.3. Compile the source again", and the remedy is to run ironwork compile on the source again. Format 0.2 was a development step (it added the directory field a module needs to run), so no release wrote it.
  • A program that INVOKEs a method now abends IRONWORK under CICS, where the method used to run with an empty handler table.
  • A program that relied on STOP RUN ending the whole task from below level one now returns to the LINKing program. A program that kept WORKING-STORAGE or EXTERNAL data across LINKs now starts fresh in each.
  • A CALL of an active program that is not RECURSIVE now ends with U4038/IGZ0064S. Add RECURSIVE to the program if the call is meant.
  • ironwork run exits 255 where a RETURN-CODE above 255 or below 0 used to exit by truncation; scripts that read the exit status should expect 255.
  • ironwork cics --serve on an address other than loopback needs --serve-public.
  • ironwork fuzz fails on a flag it does not use, where it used to ignore it.
  • ironwork check exit codes are unchanged.

Known issue

0.4.0 is on GitHub only. Its npm job failed: the release workflow named the tarball npm/portll-ironwork-0.4.0.tgz, which npm reads as a GitHub repository. crates.io and PyPI publish after npm, so they did not run, and a tag's jobs re-run with the workflow the tag holds. 0.4.1 fixes the workflow and is on npm, crates.io and PyPI; install 0.4.1 from those, or take 0.4.0's builds from this page.

Install

ironwork for COBOL 0.4.0. Builds for Linux (static), macOS and Windows, and the npm package @portll/ironwork. The ironwork-tls archives are the same builds with TLS for --sql-db (rustls, sslmode=verify-full). The Python package is on PyPI as ironwork, and the crates are on crates.io. Each file carries SLSA build provenance: gh attestation verify -R Portll/ironwork.

Assets 14
Loading