Return a CodeBlock from a custom shortcode

[albertomercurio]

Description

Hello,

I'm trying to make a custom shortcode that extends the capabilities of the built-in include shortcode, to also support labels of code blocks.

For example, I can have a file called somefile.qmd with the content

---
title: "Some Title"
---
# Some File

Some text here.

```{python}
#| label: some_label

import numpy as np

print("Hello, world!")

np.random.rand(3)
```

And a file called example.qmd with

---
title: "Include2 Example"
format: pdf
execute: 
  cache: false
---

## Heading

{{< include2 somefolder/somefile.qmd#some_label >}}

Such that only the code block is included.

I know there is the embed shortcode that does a similar thing, but here I want the code to be run from the current file, and not from the external one. For the moment the embed shortcode has several limitations.

The Lua code

In my lua code, I'm able to parse all the blocks with

blocks = quarto.utils.string_to_blocks(content)

and also to identify the block with the desired label

for _, blk in ipairs(blocks) do
  if blk.t == "CodeBlock" and blk.text:match("#|%s*label%s*:%s*" .. wanted) then
    return pandoc.Blocks{ blk }
  end
end

However, when I render the example.qmd file, quarto correctly shows the code block, but doesn't run it. I think that quarto renders the block as a plain block.

👉🏼 Link of the repo of the extension

[cscheid]

However, when I render the example.qmd file, quarto correctly shows the code block, but doesn't run it.

Running your code is responsibility of the engines, which run before Lua filters. This order is fixed and by design, because the majority of the use cases for filters involves post-processing of the execution results.

Unfortunately, that means that what you're looking to do simply isn't something that Quarto is well-equipped to let you do.

The best alternative I can think of is for you to have a pre-render script that generates your qmd from code. It's not going to be particularly nice, but I don't think there's an easy way around it.

[albertomercurio]

Hi @cscheid, thank you for you comment.

I'm wondering then how include works then. I guess that it is run before. If so, is there any possibility to add this feature to include? So that one can just include only one part of an external notebook.

[cscheid]

Include is indeed special, and is executed in TypeScript. And unfortunately no, there's no possibility to add what you're requesting for a number of reasons, including but not limited to:

1. include is intentionally unaware of syntax (otherwise, people can't use includes to create partial templates).

2. So that one can just include only one part of an external notebook.

   This is a well-meaning intent, but behind it lies a host of complicated issues around formats that this feature would need to implement in order to work well enough to be part of quarto-cli itself.

   If what you want is to include results, then embed is what you should be trying to use

   If what you want is to include executable code, then it's likely easier to decompose your notebooks into actual libraries. If you're in the jupyter world specifically, you might want to leverage something like https://nbdev.fast.ai/

[albertomercurio]

1. include is intentionally unaware of syntax (otherwise, people can't use includes to create partial templates).

My idea was also syntax-agnostic. The only requirement is to have a cell of the form

Some text...
```{lang}
#| label: my-label
a = 3+2
```

as in the embed case. My Lua extension already does it, but unfortunately only after execution.

2. If what you want is to include results, then embed is what you should be trying to use

embed would be ok, but here I want something that is executed in the current notebook and not in the external one, so that the two results are not related (for example I may have different parameters between the current notebook and the external one).

2. If you're in the jupyter world specifically, you might want to leverage something like https://nbdev.fast.ai/

My idea was to have a general way to do it, regardless the framework. Perhaps, I could try to implement a pre-render script that does it.