[FLINK-39556][table] Make file path optional in COMPILE PLAN to return plan inline as a result set

[luca-p-castelli]

What is the purpose of the change

Today COMPILE PLAN requires a file path and writes the compiled ExecNodeGraph JSON to disk:

COMPILE PLAN '/path/to/plan.json' FOR INSERT INTO sink SELECT * FROM src;

This requires the caller to share a filesystem with the executor. For tooling built on the SQL Gateway (CLIs, IDEs, CI systems) this adds filesystem-coordination complexity, and for remote SQL Gateway deployments it does not work at all — the file lands on the gateway host, not the caller's host.

This PR makes the file path optional. When omitted, the compiled plan is returned inline as a single-row, single-column STRING result set with ResultKind.SUCCESS_WITH_CONTENT — the same result-set pattern already used by EXPLAIN:

COMPILE PLAN FOR INSERT INTO sink SELECT * FROM src;
-- returns plan JSON via fetchResults

Fully backward compatible: the existing COMPILE PLAN '<path>' [IF NOT EXISTS] FOR <dml> syntax and behavior are unchanged. This is an internal-only change at the API level — both SqlCompilePlan and CompilePlanOperation are annotated @Internal; no new public API surface, no new REST endpoints.

Discussion thread: https://lists.apache.org/thread/j2z9obdyr9k0078jj1cd6tjj5nvbdk86

Brief change log

• Parser (parserImpls.ftl): make the path-and-IF NOT EXISTS block optional so the grammar accepts both COMPILE PLAN '<path>' [IF NOT EXISTS] FOR <dml> and COMPILE PLAN FOR <dml>. The two branches are LL(1)-distinguishable (FIRST sets {<QUOTED_STRING>} vs {<FOR>} are disjoint), so no explicit LOOKAHEAD directive is required.
• AST (SqlCompilePlan): planFile is now @Nullable; unparse emits the path and IF NOT EXISTS only when present.
• Operation (CompilePlanOperation): filePath is now @Nullable; constructor adds a Preconditions.checkArgument rejecting IF NOT EXISTS when no file path is specified.
• API (TableEnvironmentImpl): when the file path is null, return the plan via TableResultUtils.buildStringArrayResult("result", new String[]{compiledPlan.asJsonString()}) — the same single-column STRING SUCCESS_WITH_CONTENT shape used by other show-style commands (SHOW CREATE *, SHOW CURRENT CATALOG, etc.). Extracted a shared compileOperationToPlan helper used by both the file-based and inline paths.
• Tests: parser round-trip tests in FlinkSqlParserImplTest for inline INSERT and inline STATEMENT SET; integration tests in CompiledPlanITCase (testCompilePlanInline, testCompilePlanInlineWithStatementSet) verifying ResultKind.SUCCESS_WITH_CONTENT, the presence of flinkVersion, and the expected sink identifiers in the returned JSON.

Verifying this change

This change added tests and can be verified as follows:

• FlinkSqlParserImplTest#compilePlan — extended with parse/unparse round-trip cases for both inline forms (single INSERT, STATEMENT SET).
• CompiledPlanITCase#testCompilePlanInline — verifies single-INSERT inline form: result kind is SUCCESS_WITH_CONTENT, returned JSON contains flinkVersion, source-scan, and sink exec nodes.
• CompiledPlanITCase#testCompilePlanInlineWithStatementSet — verifies STATEMENT SET inline form: returned JSON contains both sink identifiers (sinkA, sinkB).
• All existing file-based COMPILE PLAN tests pass unchanged.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no (both SqlCompilePlan and CompilePlanOperation are @Internal)
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? yes (extends existing COMPILE PLAN syntax with an optional file path)
• If yes, how is the feature documented? JavaDocs (SqlCompilePlan and CompilePlanOperation javadocs updated to document the optional-path semantics) and SQL reference documentation.

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Claude Opus 4.7)

Generated-by: Claude Code (Claude Opus 4.7)

[luca-p-castelli]

@flinkbot run azure

[flinkbot]

CI report:

• 8fb0c22 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[luca-p-castelli]

Typo in the current docs. [IF NOT EXISTS] should be after the file path.

[spuru9]

cc @kumar-mallikarjuna