From f679efc4e5a410f7905ff7a2fe0e8a109dfd5a7d Mon Sep 17 00:00:00 2001 From: MichaelHatherly Date: Wed, 5 Mar 2025 12:09:44 +0000 Subject: [PATCH 1/4] Expand docs for `julia` engine --- docs/computations/julia.qmd | 142 ++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) diff --git a/docs/computations/julia.qmd b/docs/computations/julia.qmd index 66a21e960a..3a60641f66 100644 --- a/docs/computations/julia.qmd +++ b/docs/computations/julia.qmd @@ -318,6 +318,148 @@ The currently available options are: - `exeflags`: An array of strings which are appended to the `julia` command that starts the worker process. For example, a notebook is run with `--project=@.` by default (the environment in the directory where the notebook is stored) but this could be overridden by setting `exeflags: ["--project=/some/directory/"]`. - `env`: An array of strings where each string specifies one environment variable that is passed to the worker process. For example, `env: ["SOMEVAR=SOMEVALUE"]`. +### `juliaup` integration + +[`juliaup`](https://github.com/JuliaLang/juliaup) is the recommended way to +install and manage Julia versions. The `julia` engine supports using +`juliaup`'s `+` channel specifier to select the Julia version that a notebook +uses. This allow users to run several notebooks concurrently with different +Julia versions on the same machine without the need for customizing their +`PATH` in any way. + +To use this feature, ensure that you have used `juliaup` to install the channels +that you wish to use in your notebooks. Then add the channel specifier to the notebook +frontmatter in the `julia.exeflags` option: + +````markdown +--- +title: "A Julia 1.11 notebook" +engine: julia +julia: + exeflags: ["+1.11"] +--- + +```{{julia}} +VERSION +``` +```` + +`QuartoNotebookRunner` currently supports running Julia versions from 1.6 +onwards. Support for earlier versions is not planned. + +### Revise.jl integration + +[Revise](https://github.com/timholy/Revise.jl) allows for automatically +updating function definitions in Julia sessions. It is an essential tool in the +Julia ecosystem for any user that wishes to develop their own packages or large +projects. The `julia` engine supports using `Revise` within notebook processes +by simply importing the package into a cell at the start of a notebook. Once +imported `Revise` will automatically update functions imported from locally +developed packages in the same way as it behaves in a Julia REPL. + +Ensure that `Revise` is installed in the project environment that the notebook +is using, since the global environment is not included in the load path +provided to Julia, unlike the behaviour of a Julia REPL session. + +### Caching + +The engine has built-in support for caching notebook results. This feature is disabled by +default but can be enabled by setting `execute.cache` option to `true` in a notebook's +frontmatter: + +````markdown +--- +title: "A caching example" +engine: julia +execute: + cache: true +--- + +```{{julia}} +rand() +``` +```` + +Notebook caches are invalidated based on the following criteria: + +- Using a different Julia version to run the notebook. +- The `Manifest.toml` of the environment the notebook is run in. Adding, removing, or changing package versions will invalidate the cache. +- Changing any contents in the notebook's frontmatter. +- Changing any contents of any executable Julia cells, including cell options. + +Changes that do not invalidate a cache: + +- Editing markdown content outside of executable Julia cells. + +Caches are saved to file in a `.cache` directory alongside the notebook file. +This directory is safe to remove if you want to invalidate all caches. The +contents of the individual cache files is an internal implementation detail and +should not be relied upon. + +These caches are safe to save in CI via such tools as GitHub Actions +`action/cache` to help improve the render time of long-running notebooks that +do not often change. + +### R and Python support + +`{{r}}` and `{{python}}` executable code blocks are supported in the `julia` +engine via integrations with the +[RCall](https://github.com/JuliaInterop/RCall.jl) and +[PythonCall](https://github.com/JuliaPy/PythonCall.jl) packages respectively. +Using this feature requires the notebook author to explicitly `import` those +packages into their notebooks in a `{{julia}}` cell afterwhich they can use +`{{r}}` and `{{python}}` cells. + +````markdown +--- +title: "Multi-language notebooks" +engine: julia +--- + +```{{julia}} +import PythonCall +import RCall +``` + +Create some data in Julia: + +```{{julia}} +data = sin.(0:0.1:4pi) +``` + +Then plot it in R by interpolating the `data` variable from Julia into `R` via +the `$` syntax: + +```{{r}} +plot($data) +``` + +The same `$` syntax can be used to interpolate Julia variables into Python code as well: + +```{{python}} +len($data) +``` +```` + +### Engine "extensions" + +The implementation of `QuartoNotebookRunner` allows for extending the behaviour +of notebooks via external Julia packages. + +One example of this is the +[QuartoTools.jl](https://pumasai.github.io/QuartoTools.jl) package, which +enables fine-grained function call caching, support for serializing data +between notebook processes and normal `julia` processes, and "expandable" cell +outputs that allow for programatic creating of cell inputs and outputs within +notebooks. See the linked documentation for more thorough discussion of that +package's features. + +The same approach used by that package can be applied to other third-party +packages that wish to extend the functionality of notebooks in other ways. +Please direct questions and requests regarding this functionality to the +[QuartoNotebookRunner](https://github.com/PumasAI/QuartoNotebookRunner.jl) +repository. + ### Limitations Currently, the `engine: julia` option must be specified in each `.qmd` file. Setting the engine project-wide via `_quarto.yml` [is not yet supported](https://github.com/quarto-dev/quarto-cli/issues/3157). From 038a102c54afe2c0d9292f0f9bd4a63ecb8f5116 Mon Sep 17 00:00:00 2001 From: MichaelHatherly Date: Wed, 5 Mar 2025 17:32:37 +0000 Subject: [PATCH 2/4] Add `julia` installation note, and link to caching section --- docs/computations/julia.qmd | 7 +++++++ docs/projects/code-execution.qmd | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/computations/julia.qmd b/docs/computations/julia.qmd index 3a60641f66..0ca49bdf2c 100644 --- a/docs/computations/julia.qmd +++ b/docs/computations/julia.qmd @@ -12,6 +12,13 @@ Quarto supports executable Julia code blocks within markdown. This allows you to Quarto has two available engines for executing Julia code. The older one is using the [IJulia](https://github.com/JuliaLang/IJulia.jl) Jupyter kernel and depends on Python to run. The newer engine is using the [QuartoNotebookRunner.jl](https://github.com/PumasAI/QuartoNotebookRunner.jl/) package to render notebooks and does not have any additional dependencies beyond a Julia installation. +To use either of these engines will require that you manually install Julia if +you have not done so already. You can download it from +https://julialang.org/downloads/. There are two options for installation: the +Juliaup installation manager, and the "manual" installation approach. Unless +you know that you need to use the "manual" approach you should prefer using +Juliaup since it allows you to manage multiple Julia versions on your system. + ## Using the `jupyter` engine Below we'll describe how to [install](#installation) IJulia and related requirements but first we'll cover the basics of creating and rendering documents with Julia code blocks. diff --git a/docs/projects/code-execution.qmd b/docs/projects/code-execution.qmd index f7fac8b3f8..ceaae63f49 100644 --- a/docs/projects/code-execution.qmd +++ b/docs/projects/code-execution.qmd @@ -68,7 +68,7 @@ This sub-directory render won't use the cached `freeze` results but instead will ## Cache -You can use the `cache` option to cache the results of computations (using the [knitr cache](https://yihui.org/knitr/demo/cache/) for R documents, and [Jupyter Cache](https://jupyter-cache.readthedocs.io/en/latest/) for Jupyter documents): +You can use the `cache` option to cache the results of computations (using the [knitr cache](https://yihui.org/knitr/demo/cache/) for R documents, [Jupyter Cache](https://jupyter-cache.readthedocs.io/en/latest/) for Jupyter documents, or [`engine: julia`'s](../computations/julia.qmd#caching-1) built-in support): ``` yaml execute: From f4fdcda605e4a87bf376e0aed121cd5954434f72 Mon Sep 17 00:00:00 2001 From: Charlotte Wickham Date: Fri, 7 Mar 2025 12:39:17 -0800 Subject: [PATCH 3/4] Add julia engine highlights --- docs/computations/julia.qmd | 2 +- docs/prerelease/1.7/_highlights.qmd | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/computations/julia.qmd b/docs/computations/julia.qmd index 0ca49bdf2c..7891c3930d 100644 --- a/docs/computations/julia.qmd +++ b/docs/computations/julia.qmd @@ -368,7 +368,7 @@ Ensure that `Revise` is installed in the project environment that the notebook is using, since the global environment is not included in the load path provided to Julia, unlike the behaviour of a Julia REPL session. -### Caching +### Caching {#caching-julia} The engine has built-in support for caching notebook results. This feature is disabled by default but can be enabled by setting `execute.cache` option to `true` in a notebook's diff --git a/docs/prerelease/1.7/_highlights.qmd b/docs/prerelease/1.7/_highlights.qmd index 7471e9fcff..c9394b33cb 100644 --- a/docs/prerelease/1.7/_highlights.qmd +++ b/docs/prerelease/1.7/_highlights.qmd @@ -1 +1,8 @@ -You can view (in-progress) documentation for the next version of Quarto, v1.7, on our pre-release documentation site, [prerelease.quarto.org](https://prerelease.quarto.org), including a list of [highlights](https://prerelease.quarto.org/docs/prerelease/1.7/). \ No newline at end of file +Quarto 1.7 includes the following new features: + +- Improvements to the `julia` engine: + - [`juliaup` integration](/docs/computations/julia.qmd#juliaup-integration): Use specific versions of Julia in your notebooks. + - [R and Python support](/docs/computations/julia.qmd#r-and-python-support): Include `{r}` and `{python}` executable code cells via the RCall and PythonCall packages. + - [Caching](/docs/computations/julia.qmd#caching-julia): Save time rendering long-running notebooks by caching results. + - [Revise.jl integration](/docs/computations/julia.qmd#revise.jl-integration): Automatically update function definitions in Julia sessions. + From 0be6327d32b0470e312aaad8898285adf31a854a Mon Sep 17 00:00:00 2001 From: Michael Hatherly Date: Mon, 10 Mar 2025 08:56:15 +0000 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: Charlotte Wickham --- docs/computations/julia.qmd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/computations/julia.qmd b/docs/computations/julia.qmd index 7891c3930d..d8fae36377 100644 --- a/docs/computations/julia.qmd +++ b/docs/computations/julia.qmd @@ -12,11 +12,11 @@ Quarto supports executable Julia code blocks within markdown. This allows you to Quarto has two available engines for executing Julia code. The older one is using the [IJulia](https://github.com/JuliaLang/IJulia.jl) Jupyter kernel and depends on Python to run. The newer engine is using the [QuartoNotebookRunner.jl](https://github.com/PumasAI/QuartoNotebookRunner.jl/) package to render notebooks and does not have any additional dependencies beyond a Julia installation. -To use either of these engines will require that you manually install Julia if +Using either of these engines will require manually installing Julia if you have not done so already. You can download it from https://julialang.org/downloads/. There are two options for installation: the -Juliaup installation manager, and the "manual" installation approach. Unless -you know that you need to use the "manual" approach you should prefer using +Juliaup installation manager, or the "manual" installation approach. Unless +you know that you need to use the "manual" approach, use Juliaup since it allows you to manage multiple Julia versions on your system. ## Using the `jupyter` engine @@ -371,7 +371,7 @@ provided to Julia, unlike the behaviour of a Julia REPL session. ### Caching {#caching-julia} The engine has built-in support for caching notebook results. This feature is disabled by -default but can be enabled by setting `execute.cache` option to `true` in a notebook's +default but can be enabled by setting the `execute.cache` option to `true` in a notebook's frontmatter: ````markdown @@ -390,7 +390,7 @@ rand() Notebook caches are invalidated based on the following criteria: - Using a different Julia version to run the notebook. -- The `Manifest.toml` of the environment the notebook is run in. Adding, removing, or changing package versions will invalidate the cache. +- Changes to the `Manifest.toml` of the environment the notebook is run in. Adding, removing, or changing package versions will invalidate the cache. - Changing any contents in the notebook's frontmatter. - Changing any contents of any executable Julia cells, including cell options.