Fetch and expand copybooks through Zowe CLI#4
Merged
Merged
Conversation
Let cq fetch copybooks as text and stream record data in binary mode through the installed Zowe CLI, avoiding shell-specific process substitution and PowerShell byte-pipeline hazards. Constraint: Reuse the user's existing Zowe installation and configuration; do not add an SDK dependency. Rejected: Shell wrappers and temporary files | they remain platform-specific and prevent direct streaming Confidence: high Scope-risk: moderate Directive: Keep copybook Zowe reads in text mode and record-data reads in binary mode. Tested: GOCACHE=/tmp/cq-go-build go test ./...; GOCACHE=/tmp/cq-go-build go test -race ./...; GOCACHE=/tmp/cq-go-build go vet ./...; Windows amd64 build and test-binary cross-compilation
Expand plain COPY member statements before parsing and resolve each member recursively through a configurable, cached Zowe search hierarchy. Constraint: Configuration uses DSNSearchPath in cq.json or an explicit --config file. Rejected: Implicitly search only the root copybook library | enterprise copybooks commonly span several ordered libraries Rejected: Support COPY REPLACING immediately | token substitution semantics need a separate tested contract Confidence: high Scope-risk: moderate Directive: Preserve first-success DSN ordering and report COPY cycles with the full member chain. Tested: GOCACHE=/tmp/cq-go-build go test ./...; GOCACHE=/tmp/cq-go-build go test -race ./...; GOCACHE=/tmp/cq-go-build go vet ./...; Windows amd64 application and test-binary cross-compilation
Resolve implicit cq configuration through os.UserConfigDir so Linux, macOS, and Windows use established per-user locations while retaining --config overrides. Constraint: Missing implicit config remains optional until nested COPY lookup is needed. Rejected: Search the working directory for cq.json | surprising behavior in arbitrary project directories Confidence: high Scope-risk: narrow Directive: Keep platform defaults derived from os.UserConfigDir. Tested: GOCACHE=/tmp/cq-go-cache go test ./...; go test -race ./...; go vet ./...; Windows amd64 CLI and test-binary cross-compilation
Remove the explicit config override so nested COPY resolution has one predictable platform-standard configuration source and a smaller CLI surface. Constraint: DSNSearchPath remains optional until a COPY statement needs resolution. Rejected: Retain an internal arbitrary-path loader | it preserves unused complexity after removing the flag Confidence: high Scope-risk: narrow Directive: Keep cq configuration rooted at os.UserConfigDir unless a concrete multi-config use case emerges. Tested: go test ./...; go test -race ./...; go vet ./...; CLI help smoke test; Windows amd64 CLI and test-binary cross-compilation
Add cq config so users can create and open the sole configuration file without locating platform-specific directories manually. Constraint: Launch editors without a shell and preserve existing configuration content. Rejected: Reintroduce a config-path flag | it would undo the single-path CLI simplification Confidence: high Scope-risk: narrow Directive: Keep cq config aligned with defaultConfigFile and never overwrite an existing config. Tested: go test ./...; go test -race ./...; go vet ./...; cq config creation/editor smoke test; CLI help smoke test; Windows amd64 CLI and test-binary cross-compilation
Detect a non-comment PROCEDURE DIVISION in the primary input so whole COBOL programs fail clearly without triggering COPY resolution. Constraint: Reuse lexer normalization so fixed/free comments and quoted literals do not produce false positives. Rejected: Validate every recursively fetched member | broader nested-member policy than the requested input guard Confidence: high Scope-risk: narrow Directive: Keep program-source detection ahead of the first resolver call. Tested: go test ./...; go test -race ./...; go vet ./...; Windows amd64 CLI and test-binary cross-compilation
State that cq neither bundles Zowe CLI nor has Linux Foundation or Zowe project affiliation, while acknowledging the registered trademark. Constraint: Keep the notice adjacent to the Zowe invocation documentation. Confidence: high Scope-risk: narrow Directive: Revisit licensing and notices before bundling any Zowe artifacts. Tested: git diff --check; documentation-only change
Use Zowe download-to-file for record data so packed and binary fields reach cq unchanged and Zowe failures are known before JSON output begins. Constraint: Correct byte preservation takes precedence over streaming; temporary downloads are removed after decoding. Rejected: zowe zos-files view data-set --binary | observed to corrupt packed-decimal bytes and conflict with profile encoding Confidence: high Scope-risk: moderate Directive: Do not restore the Zowe view path for binary record data without byte-for-byte mainframe evidence. Tested: go test ./...; go test -race ./...; go vet ./...; CLI help smoke test; Windows amd64 CLI and test-binary cross-compilation Not-tested: Live Zowe profile and z/OSMF transfer are unavailable in this workspace.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Zowe transfer behavior
Copybooks use the text view path. Record data uses the byte-preserving binary download path, which gives binary mode precedence over profile encoding. cq waits for a successful download before producing JSON and removes the temporary file after decoding. Manual stdin pipelines cannot expose producer failure to cq, so the README documents pipefail and warns against view --binary.
Configuration
The file is config.json under the cq directory returned by Go os.UserConfigDir (XDG config on Linux, Application Support on macOS, and AppData on Windows). cq config creates a minimal file when absent, preserves existing content, and uses VISUAL, EDITOR, or the platform fallback editor.
Validation
Live Zowe/z/OSMF transfer could not be exercised in the development workspace; the change is based on user mainframe evidence and the current official Zowe download command/SDK implementation.
Current COPY scope
Simple COPY MEMBER. statements are supported. COPY REPLACING and COPY OF/IN clauses return explicit errors.