Fix list-valued generate/analyse on parallel workers (future + socket)#91
Merged
philchalmers merged 3 commits intoJun 24, 2026
Merged
Conversation
…ckends A list-valued `generate` aborted under `parallel = TRUE` with "object 'GENERATE_FUNCTIONS' not found": runSimulation() exported ANALYSE_FUNCTIONS to the cluster workers but never GENERATE_FUNCTIONS, so combined_Generate() could not find the function list on the workers. Introduce a local GENERATE_FUNCTIONS (mirroring ANALYSE_FUNCTIONS) and add a guarded clusterExport() for it. Also expand the regression test to exercise both list-valued generate and list-valued analyse across the serial, socket (parallel = TRUE) and future (multisession) backends, covering this fix and the earlier future-backend fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
philchalmers
left a comment
Owner
There was a problem hiding this comment.
It's awkward to do the redefinition of mainsim, but I don't see away around this since future doesn't expose the cluster info. Could you add this information to the NEWS.md file?
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Hi @philchalmers , I asked Claude to give it a shot. |
philchalmers
approved these changes
Jun 24, 2026
Owner
|
Good enough for me. I'll make any edits after the merge. Thanks! |
philchalmers
added a commit
that referenced
this pull request
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix list-valued generate/analyse on parallel workers (future + socket)
Claude wrote this, I don't have the understanding of how parallel processing in R works to review this, sorry. But I thought perhaps it might save you time by having Claude's proposed changes + tests.
Closes #90.
What
Make list-valued
generate/analysework on every back-end. Two gaps caused fatalterminations when the function lists (stashed in
SimDesign:::.SIMDENV) were notdelivered to parallel workers:
parallel = "future"— neitherGENERATE_FUNCTIONSnorANALYSE_FUNCTIONSreached the workers (
future's automatic global detection does not exportpackage-internal objects), so the run aborted with
object 'ANALYSE_FUNCTIONS' not found(or'GENERATE_FUNCTIONS').parallel = TRUE(socket/MPI) — a list-valuedgenerateaborted withobject 'GENERATE_FUNCTIONS' not found, becauseclusterExport()shippedANALYSE_FUNCTIONSbut neverGENERATE_FUNCTIONS.Serial (
parallel = FALSE) was always fine.Why
When
generate/analyseare lists,runSimulation()substitutes the internalcombined_Generate()/combined_Analyses()dispatchers and stores the originalfunction lists in the package-internal
.SIMDENV. Workers load a fresh copy of thepackage whose
.SIMDENVis empty, so the lists must be sent to them explicitly. Onelist was missing from the socket export, and the
futurepath did not export them atall.
How
R/runSimulation.R: add a localGENERATE_FUNCTIONS(mirroringANALYSE_FUNCTIONS)and a guarded
clusterExport(cl, "GENERATE_FUNCTIONS").R/analysis.R: in thefuturebranch ofAnalysis(), wrap the worker function soevery future restores the captured function lists into the worker's
.SIMDENVviagetFromNamespace()before the replication runs — correct for anyfutureplan.Tests
tests/tests/test-01-core.Rnow exercises list-valuedgenerateand list-valuedanalyseacross serial, socket (parallel = TRUE) andfuture(multisession)back-ends, asserting no
FATAL_TERMINATION.Verification
Built and installed the patched package; both reprexes return
FATAL_TERMINATION = NULLand correct output columns across 3/3 trials on each affected back-end. The samereprexes fail deterministically (3/3) on the released package.