Python RPC: don't send prefix/markers for delegating wrapper types#7424
Merged
knutwannheden merged 1 commit intomainfrom Apr 18, 2026
Merged
Python RPC: don't send prefix/markers for delegating wrapper types#7424knutwannheden merged 1 commit intomainfrom
knutwannheden merged 1 commit intomainfrom
Conversation
`Py.ExpressionStatement` and `Py.StatementExpression` delegate `prefix` and `markers` to their wrapped child. The Java receiver and the Python sender already skip them in `preVisit`, but the Java sender was emitting them anyway. The Python receiver tried to drain those extra messages with `q.receive(None)`, which only consumes the header — not the nested `Space` comments-list and whitespace messages. The queue then desynchronized, and a later `receive_list_defined` for `Space.comments` under a nested node (typically the expression under a `Py.Await`) hit a `NO_CHANGE` where it expected the positions array, producing: Expected positions array but got: RpcObjectData(state=NO_CHANGE, value=None, ...) The failure only surfaced for CHANGE (recipe-modified tree), not ADD, because on ADD the receiver can recover via the `valueType` on the header. Flagship recipes like `Common static analysis issues` regularly hit this on Python sources. Align the Java sender with the existing contract: send only `id` for `Py.ExpressionStatement` / `Py.StatementExpression`. Simplify the Python receiver to match. A new round-trip test reproduces the production signature (fails without the Java sender change).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Flagship recipe runs (
Common static analysis issues) were failing on every Python repository with the following signature propagated back from the Python RPC server:Affected Apr 17 runs included 8
finos/legend-juju-*/finos/symphony-bdk-pythonrepos plusNetflix/metaflow.Root cause
Py.ExpressionStatementandPy.StatementExpressiondelegateprefixandmarkersto their wrapped child, so those fields should be serialized via the child'spreVisit, not the wrapper's. Three of the four sides already honored this:PythonReceiver.preVisit— only readsidfor these types.PythonRpcSender._visit— only sendsid.PythonRpcReceiver._visit— attempted to drain them viaq.receive(None).But Java
PythonSender.preVisitunconditionally emittedid + prefix + markersfor everyJ. The Python receiver'sq.receive(None)consumed only the header message, not the nestedSpacecomments-list and whitespace messages. The queue then desynchronized, and a laterreceive_list_definedforSpace.comments(often underPy.Await.expressionas in the stack above) saw aNO_CHANGEwhere it expected the positions array.The failure only surfaced for CHANGE (recipe-modified tree), not ADD, because on ADD the receiver can recover via the
valueTypeon the header and instantiate aSpaceto drain via its codec. CHANGE omitsvalueTypeso no codec runs.Summary
PythonSender.preVisit: skipprefix/markersforPy.ExpressionStatementandPy.StatementExpression, matching the receiver contract and the Python sender.python_receiver.py::_visit: simplify the wrapper-type branch to only readid, since the sender no longer emits the extra messages.PythonSenderReceiverRoundTripTestreproducing the production signature via aCHANGEround trip of anExpressionStatement(Await(Identifier))with a modified prefix — fails without thePythonSenderchange with the sameExpected CHANGE with positions in receiveListerror.Test plan
./gradlew :rewrite-python:test --tests PythonSenderReceiverRoundTripTest— passes with the fix, fails without it (verified by reverting the sender change)../gradlew :rewrite-python:test— full Java test suite green.uv run pytest tests/ --timeout=60 --ignore=tests/rpc— full Python unit suite (1157 tests) green.