Update bundled Julia engine extension to 0.2.1

news/changelog-1.11.md

@@ -28,6 +28,14 @@ All changes included in 1.11:
 
 - ([#14735](https://github.com/quarto-dev/quarto-cli/issues/14735)): Fix `cache-globals` rejecting arrays and booleans, so it now accepts the same forms as `cache-vars` and as knitr itself. Previously only a single string validated, which rejected both `cache-globals: [var_1, var_2]` and the documented `cache-globals: false`.
 
+### `julia`
+
+- ([#14834](https://github.com/quarto-dev/quarto-cli/issues/14834)): Fix `ERROR: Internal Error` when rendering documents whose payload exceeds the Julia server's socket send buffer.
+- ([PumasAI/quarto-julia-engine#8](https://github.com/PumasAI/quarto-julia-engine/pull/8)): Support `keep-ipynb`, which writes the executed notebook to `<stem>.ipynb` alongside the source file.
+- ([PumasAI/quarto-julia-engine#13](https://github.com/PumasAI/quarto-julia-engine/pull/13)): Shell (`;`), help (`?`), and Pkg (`]`) mode cells now work when the cell has `#|` options.
+- ([PumasAI/quarto-julia-engine#11](https://github.com/PumasAI/quarto-julia-engine/pull/11)): Support `fig-format: retina`, normalized to `png` with doubled `fig-dpi` as in the `jupyter` and `knitr` engines.
+- ([PumasAI/quarto-julia-engine#7](https://github.com/PumasAI/quarto-julia-engine/pull/7)): Support `execute-dir`, shared worker processes across notebooks with matching configs (`share_worker_process: true`), and cached worker environments that persist across sessions.
+
 ## Other fixes and improvements
 
 - ([#14775](https://github.com/quarto-dev/quarto-cli/issues/14775)): Fix a crash when the `QUARTO_R` environment variable is set to a malformed path. Quarto now warns and falls back to other R lookup strategies instead of aborting the render. ([posit-dev/positron#15614](https://github.com/posit-dev/positron/discussions/15614))

src/resources/extension-subtrees/julia-engine/.github/workflows/EnforceChangelog.yml

@@ -0,0 +1,14 @@
+name: "Enforce changelog"
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
+
+jobs:
+  changelog:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+      - uses: dangoslen/changelog-enforcer@204e7d3ef26579f4cd0fd759c57032656fdf23c7 # v3.6.1
+        with:
+          changeLogPath: 'CHANGELOG.md'
+          skipLabels: 'skip-changelog'

src/resources/extension-subtrees/julia-engine/AGENTS.md

@@ -50,6 +50,17 @@ quarto render scratch/test.qmd
 
 Since this directory is a quarto project with the extension in `_extensions/`, quarto discovers and uses the engine from here.
 
+### Do not use a checkout under a dot-directory
+
+Quarto skips every path containing a dot-directory when collecting project input
+files (`kSkipHidden` in quarto-cli's `src/core/path.ts`, matched against absolute
+paths). A checkout under e.g. `.claude/worktrees/` therefore has no project input
+files at all, so each render falls back to the single-file path, which looks for
+extensions starting at the document's own directory. It finds quarto's bundled
+subtree copy of this engine instead of `_extensions/` here, and renders and tests
+then exercise the shipped engine rather than the working tree, without any
+warning. Work in a checkout whose absolute path contains no dot-directory.
+
 ## CI
 
 CI runs on all three platforms (Linux, macOS, Windows) against a pinned quarto-cli revision (see `QUARTO_CLI_REV` in `.github/workflows/ci.yml`). It:
@@ -59,3 +70,7 @@ CI runs on all three platforms (Linux, macOS, Windows) against a pinned quarto-c
 3. Runs the full test suite
 
 When bumping `QUARTO_CLI_REV`, use the full commit hash annotated with the version tag for clarity (e.g. `abc123 # v1.9.35`).
+
+## Changelog
+
+Every PR with user-facing or otherwise meaningful changes must include an update to `CHANGELOG.md` (enforced by CI). Add entries under the `## Unreleased` section. Use the `skip-changelog` label to bypass the check for PRs that don't need an entry (e.g. internal cleanups, CI, or docs changes).

src/resources/extension-subtrees/julia-engine/CHANGELOG.md

@@ -0,0 +1,34 @@
+# Quarto Julia Engine changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## Unreleased
+
+## [0.2.1](https://github.com/PumasAI/quarto-julia-engine/releases/tag/v0.2.1) - 2026-09-02
+
+- Updated to QuartoNotebookRunner 0.18.2 [#13](https://github.com/PumasAI/quarto-julia-engine/pull/13):
+  - Shell (`;`), help (`?`), and Pkg (`]`) mode cells now work when the cell has `#|` options.
+- Large documents no longer fail with `ERROR: Internal Error` when the payload sent to the Julia server exceeds the socket send buffer ([quarto-cli#14834](https://github.com/quarto-dev/quarto-cli/issues/14834), [#12](https://github.com/PumasAI/quarto-julia-engine/pull/12)).
+
+## [0.2.0](https://github.com/PumasAI/quarto-julia-engine/releases/tag/v0.2.0) - 2026-03-24
+
+- Added `keep-ipynb` support. When `keep-ipynb: true` is set in document YAML, the executed notebook is written to `<stem>.ipynb` alongside the source file [#8](https://github.com/PumasAI/quarto-julia-engine/pull/8).
+- Updated to QuartoNotebookRunner 0.18.1 [#11](https://github.com/PumasAI/quarto-julia-engine/pull/11):
+  - **`fig-format: retina` support**: Normalized to `png` with doubled `fig-dpi`, matching Quarto's Jupyter and knitr backends. Fixes blurry plots with default HTML settings.
+  - **Improved cache invalidation**: Cache key now hashes full `Manifest.toml` content (SHA-224) so dependency version changes correctly invalidate the cache.
+- Updated to QuartoNotebookRunner 0.18.0 [#7](https://github.com/PumasAI/quarto-julia-engine/pull/7):
+  - **Shared worker processes**: Multiple notebooks can share a single worker process when their configs match. Enable with `share_worker_process: true`.
+  - **`execute-dir` support**: Quarto's `execute-dir` option (e.g. `execute-dir: project`) is now passed through correctly.
+  - **Cached worker environments**: Worker envs persist across sessions, skipping environment setup on subsequent runs when Julia and QNR versions haven't changed.
+  - **Relaxed manifest version checking**: Only major.minor is checked by default, matching Julia's Pkg behavior. Opt into strict major.minor.patch via `julia: strict_manifest_versions: true`.
+  - **Diagnostic file logging**: Set `QUARTONOTEBOOKRUNNER_LOG` to a directory path to enable timestamped log files.
+  - **Echo handling for Python/R cells**: Fixed duplicate YAML keys and added proper support for `echo: false` and `echo: fenced`.
+
+## [0.1.0](https://github.com/PumasAI/quarto-julia-engine/releases/tag/v0.1.0) - 2026-03-19
+
+- Initial release as a standalone Quarto extension, extracted from [quarto-cli](https://github.com/quarto-dev/quarto-cli).
+- Bundled Julia resource files (`Project.toml`, `ensure_environment.jl`, `quartonotebookrunner.jl`, `start_quartonotebookrunner_detached.jl`) into the extension, making it fully self-contained [#6](https://github.com/PumasAI/quarto-julia-engine/pull/6).
+- Added CI workflow with tests on Linux, macOS, and Windows [#6](https://github.com/PumasAI/quarto-julia-engine/pull/6).

src/resources/extension-subtrees/julia-engine/_extensions/julia-engine/Project.toml

@@ -2,4 +2,4 @@
 QuartoNotebookRunner = "4c0109c6-14e9-4c88-93f0-2b974d3468f4"
 
 [compat]
-QuartoNotebookRunner = "=0.17.4"
+QuartoNotebookRunner = "=0.18.2"

src/resources/extension-subtrees/julia-engine/_extensions/julia-engine/_extension.yml

@@ -1,5 +1,5 @@
 title: Quarto Julia Engine Extension
-version: 0.1.0
+version: 0.2.1
 quarto-required: ">=1.9.0"
 contributes:
   engines:

src/resources/extension-subtrees/julia-engine/_extensions/julia-engine/julia-engine.js

@@ -621,6 +621,14 @@ function encodeBase64(data) {
   return new TextDecoder().decode(output);
 }
 
+// deno:https://jsr.io/@std/io/0.224.0/write_all.ts
+async function writeAll(writer, data) {
+  let nwritten = 0;
+  while (nwritten < data.length) {
+    nwritten += await writer.write(data.subarray(nwritten));
+  }
+}
+
 // src/constants.ts
 var kJuliaEngine = "julia";
 var kExecuteDaemon = "daemon";
@@ -631,6 +639,7 @@ var kFigFormat = "fig-format";
 var kFigPos = "fig-pos";
 var kIpynbProduceSourceNotebook = "produce-source-notebook";
 var kKeepHidden = "keep-hidden";
+var kKeepIpynb = "keep-ipynb";
 
 // src/julia-engine.ts
 var isWindows2 = Deno.build.os === "windows";
@@ -749,6 +758,11 @@ var juliaEngineDiscovery = {
           language: "julia"
         };
         const assets = quarto.jupyter.assets(options.target.input, options.format.pandoc.to);
+        if (options.format.execute[kKeepIpynb]) {
+          const stem = options.target.source.replace(/\.[^.]+$/, "");
+          const ipynbPath = stem + ".ipynb";
+          Deno.writeTextFileSync(ipynbPath, JSON.stringify(nb, null, 2));
+        }
         const result = await quarto.jupyter.toMarkdown(nb, {
           executeOptions: options,
           language: nb.metadata.kernelspec.language.toLowerCase(),
@@ -1150,10 +1164,7 @@ async function writeJuliaCommand(conn, command, secret, options, onProgressUpdat
   }) + "\n";
   const messageBytes = new TextEncoder().encode(message);
   trace(options, `write command "${command.type}" to socket server`);
-  const bytesWritten = await conn.write(messageBytes);
-  if (bytesWritten !== messageBytes.length) {
-    throw new Error("Internal Error");
-  }
+  await writeAll(conn, messageBytes);
   let restOfPreviousResponse = new Uint8Array(512);
   let restLength = 0;
   while (true) {

src/resources/extension-subtrees/julia-engine/src/constants.ts

@@ -13,3 +13,4 @@ export const kFigFormat = "fig-format";
 export const kFigPos = "fig-pos";
 export const kIpynbProduceSourceNotebook = "produce-source-notebook";
 export const kKeepHidden = "keep-hidden";
+export const kKeepIpynb = "keep-ipynb";

src/resources/extension-subtrees/julia-engine/src/julia-engine.ts

@@ -8,6 +8,7 @@
 import { dirname, fromFileUrl, join, resolve } from "path";
 import { existsSync } from "fs/exists";
 import { encodeBase64 } from "encoding/base64";
+import { writeAll } from "jsr:@std/io@0.224.0/write-all";
 
 // Type imports from Quarto via import map
 import type {
@@ -37,6 +38,7 @@ import {
   kIpynbProduceSourceNotebook,
   kJuliaEngine,
   kKeepHidden,
+  kKeepIpynb,
 } from "./constants.ts";
 
 // Platform detection
@@ -220,6 +222,14 @@ export const juliaEngineDiscovery: ExecutionEngineDiscovery = {
           options.format.pandoc.to,
         );
 
+        // Write notebook to file if keep-ipynb is set (must happen before
+        // toMarkdown which mutates nb in place)
+        if (options.format.execute[kKeepIpynb]) {
+          const stem = options.target.source.replace(/\.[^.]+$/, "");
+          const ipynbPath = stem + ".ipynb";
+          Deno.writeTextFileSync(ipynbPath, JSON.stringify(nb, null, 2));
+        }
+
         // NOTE: for perforance reasons the 'nb' is mutated in place
         // by jupyterToMarkdown (we don't want to make a copy of a
         // potentially very large notebook) so should not be relied
@@ -832,10 +842,7 @@ async function writeJuliaCommand<T extends ServerCommand["type"]>(
   const messageBytes = new TextEncoder().encode(message);
 
   trace(options, `write command "${command.type}" to socket server`);
-  const bytesWritten = await conn.write(messageBytes);
-  if (bytesWritten !== messageBytes.length) {
-    throw new Error("Internal Error");
-  }
+  await writeAll(conn, messageBytes);
 
   // a string of bytes received from the server could start with a
   // partial message, contain multiple complete messages (separated by newlines) after that

src/resources/extension-subtrees/julia-engine/tests/docs/julia-engine/.gitignore

@@ -4,3 +4,7 @@
 # Allow only source files
 !*/*.qmd
 !*/_quarto.yml
+!*/_metadata.yml
+
+# Generated by the large payload test
+large-payload/*.qmd

src/resources/extension-subtrees/julia-engine/tests/docs/julia-engine/engine-reordering/_metadata.yml

@@ -0,0 +1 @@
+engines: ["julia"]

src/resources/extension-subtrees/julia-engine/tests/docs/julia-engine/keep-ipynb/keep-ipynb.qmd

@@ -0,0 +1,10 @@
+---
+title: keep-ipynb test
+engine: julia
+format: html
+keep-ipynb: true
+---
+
+```{julia}
+1 + 1
+```

src/resources/extension-subtrees/julia-engine/tests/docs/julia-engine/keep-ipynb/no-keep-ipynb.qmd

@@ -0,0 +1,9 @@
+---
+title: no keep-ipynb test
+engine: julia
+format: html
+---
+
+```{julia}
+1 + 1
+```

src/resources/extension-subtrees/julia-engine/tests/smoke/julia-engine/render.test.ts

@@ -44,6 +44,40 @@ Deno.test("source ranges with includes", async () => {
   try { Deno.removeSync(outputFile); } catch { /* ok */ }
 });
 
+Deno.test("keep-ipynb", async () => {
+  const dir = docs("julia-engine/keep-ipynb");
+  const input = join(dir, "keep-ipynb.qmd");
+  renderQmd(input, ["--to", "html"]);
+
+  const outputHtml = join(dir, "keep-ipynb.html");
+  assert(existsSync(outputHtml), `Output file ${outputHtml} should exist`);
+
+  const ipynbFile = join(dir, "keep-ipynb.ipynb");
+  assert(existsSync(ipynbFile), `keep-ipynb file ${ipynbFile} should exist`);
+
+  const content = await Deno.readTextFile(ipynbFile);
+  const nb = JSON.parse(content);
+  assert(nb.cells, "Notebook should have cells");
+  assert(nb.metadata, "Notebook should have metadata");
+
+  try { Deno.removeSync(outputHtml); } catch { /* ok */ }
+  try { Deno.removeSync(ipynbFile); } catch { /* ok */ }
+});
+
+Deno.test("no keep-ipynb by default", () => {
+  const dir = docs("julia-engine/keep-ipynb");
+  const input = join(dir, "no-keep-ipynb.qmd");
+  renderQmd(input, ["--to", "html"]);
+
+  const outputHtml = join(dir, "no-keep-ipynb.html");
+  assert(existsSync(outputHtml), `Output file ${outputHtml} should exist`);
+
+  const ipynbFile = join(dir, "no-keep-ipynb.ipynb");
+  assert(!existsSync(ipynbFile), `ipynb file ${ipynbFile} should NOT exist without keep-ipynb`);
+
+  try { Deno.removeSync(outputHtml); } catch { /* ok */ }
+});
+
 Deno.test("engine reordering", () => {
   const dir = docs("julia-engine/engine-reordering");
   renderQmd("notebook.qmd", ["--to", "html"], dir);
@@ -53,3 +87,28 @@ Deno.test("engine reordering", () => {
 
   try { Deno.removeSync(outputFile); } catch { /* ok */ }
 });
+
+Deno.test("large payload exceeding the socket send buffer", () => {
+  const dir = docs("julia-engine/large-payload");
+  const input = join(dir, "large-payload.qmd");
+
+  const line =
+    '<span class="x">Lorem ipsum dolor sit amet, consectetur adipiscing elit.</span>';
+  const body = new Array(25000).fill(line).join("\n");
+  Deno.mkdirSync(dir, { recursive: true });
+  Deno.writeTextFileSync(
+    input,
+    "---\ntitle: large payload\nformat: markdown\nengine: julia\n---\n\n" +
+      '```{julia}\n#| echo: false\n"marker-" * string(1 + 1)\n```\n\n' +
+      "```{=html}\n" + body + "\n```\n",
+  );
+
+  renderQmd(input, ["--to", "markdown"]);
+
+  const outputFile = join(dir, "large-payload.md");
+  assert(existsSync(outputFile), `Output file ${outputFile} should exist`);
+  assertMatch(Deno.readTextFileSync(outputFile), /marker-2/);
+
+  try { Deno.removeSync(outputFile); } catch { /* ok */ }
+  try { Deno.removeSync(input); } catch { /* ok */ }
+});