Tests are not hermetic: several suites require live model downloads

[Rishabh060105]

## Title

Tests are not hermetic: several suites require live model downloads

## Bug Report

### Summary

Some tests in this repository are not hermetic: they require live downloads of external Concerto model files at runtime and fail when network access is unavailable.

I verified this locally by running the affected suites directly in the repository. The failures are reproducible and are caused by remote model fetches rather than local logic regressions.

---

## Reproduction

From the repository root:

```bash
node --experimental-vm-modules node_modules/jest-cli/bin/jest.js --runInBand --coverage=false test/TemplateMarkInterpreter.test.ts

node --experimental-vm-modules node_modules/jest-cli/bin/jest.js --runInBand --coverage=false test/TemplateArchiveProcessor.test.ts

---

Observed Behavior

test/TemplateMarkInterpreter.test.ts

Fails on templates that depend on imported external models.

The failures occur at:

await modelManager.updateExternalModels();

in test/TemplateMarkInterpreter.test.ts.

Example error:

Failed to load model file.
Job: https://models.accordproject.org/money@0.3.0.cto
Details: TypeError: fetch failed

test/TemplateArchiveProcessor.test.ts

Also fails with remote model fetch errors such as:

Failed to load model file.
Job: https://models.accordproject.org/accordproject/contract@0.2.0.cto
Details: TypeError: fetch failed

---

What I Verified Locally

Known-good suites pass

• test/HelloWorld.test.ts
• test/GenerateOptions.test.ts

Network-dependent suites fail

test/TemplateMarkInterpreter.test.ts

• 24 tests passed
• 7 tests failed

test/TemplateArchiveProcessor.test.ts

• 1 test passed
• 6 tests failed

This suggests the problem is specifically test hermeticity rather than a general package regression.

---

Concrete Sources of External Dependency

Examples of fixtures that import remote .cto files:

test/templates/good/insurance-premium/model.cto

import org.accordproject.money@0.3.0.{MonetaryAmount}
from https://models.accordproject.org/money@0.3.0.cto

test/templates/good/optional-nested/model.cto

Imports remote:

• money models
• time models
• party models

test/archives/latedeliveryandpenalty-typescript/model/model.cto

Imports remote:

• time models
• contract models
• runtime models

---

Expected Behavior

The test suite should be runnable without live network access.

Tests should resolve model dependencies from checked-in fixtures, mocked downloads, or another offline-safe mechanism so that CI and local development remain reproducible.

---

Why This Matters

• Improves CI reliability in sandboxed or restricted environments
• Makes local test runs reproducible
• Avoids false negatives caused by network availability rather than code changes

---

Possible Fixes

• Vendor the required external .cto models into test fixtures
• Remove updateExternalModels() from tests where offline fixtures can be used
• Mock or stub external model downloads in tests
• Ensure archive test fixtures fully resolve dependencies offline

---

Additional Note

Some archive tests already use:

Template.fromDirectory(..., { offline: true })

However, there are still test paths that trigger remote fetches, so offline coverage is currently incomplete.

[Rishabh060105]

Hi @mttrbrts,

I investigated the test failures in template-engine and found that some of the test suites are not fully hermetic. In particular, a subset of tests still depends on live downloads of external Concerto .cto models, so they fail in offline or sandboxed environments even when the local code is fine.

I verified this locally:

• test/HelloWorld.test.ts passes
• test/GenerateOptions.test.ts passes
• test/TemplateMarkInterpreter.test.ts fails only on templates that call updateExternalModels() and import remote models
• test/TemplateArchiveProcessor.test.ts also fails on remote model fetches

The issue seems to be test reproducibility rather than a regression in the runtime itself. Likely fixes would be vendoring the required .cto models into fixtures, mocking external downloads in tests, or otherwise ensuring those test paths run fully offline.

I drafted an issue for this under the title:

Tests are not hermetic: several suites require live model downloads

Happy to help turn that into a PR if that would be useful.

[mttrbrts]

Good catch @Rishabh060105, we have a pattern for this I'm the Concerto tests that you can follow. External models are cached with an @ prefix. You're very welcome to put in a PR.