allow Vim script expressions to be evaluated inside heredocs

[lacygoill]

Is your feature request about something that is currently impossible or hard to do? Please describe the problem.

It is not easy to include the value of a Vim script expression inside a heredoc:

vim9script
var lines =<< trim END
    echo $HOME
END
echo lines

['echo $HOME']

If we want $HOME to be evaluated, we need an extra call to a combination of map() and substitute():

vim9script
var lines =<< trim END
    echo $HOME
END
lines->map((_, line) => line->substitute('$HOME', $HOME, ''))
echo lines

[echo /home/user]

Which makes the code less readable, and is probably not always reliable in the general case (i.e. substitute() might replace tokens which we want to be left alone).

Describe the solution you'd like

Extend the backtick expansion syntax which supports Vim script expressions (the one documented at :help E1083) to heredocs:

:e `=tempname()`
   ^^          ^

With this feature, the previous snippet could be simplified into this:

vim9script
var lines =<< trim eval END
    echo `=$HOME`
END
echo lines

Notice the presence of the new eval argument on the first line of the assigment:

var lines =<< trim eval END
                   ^--^

It would be necessary to avoid breaking existing scripts. With eval, Vim would look for Vim script expressions to evaluate.

Describe alternatives you've considered

An alternative is to continue using an extra call to a combination of map() and substitute(), which – again – is less readable and less reliable than the proposed syntax.

Additional context

In bash, there are 2 kinds of heredocs. The ones which do not try to expand any parameter:

cat <<'EOF'
    echo $HOME
EOF

echo $HOME

And the ones which do:

cat <<EOF
    echo $HOME
EOF

echo /home/user

The difference between the 2 depends on whether the end word is quoted. Currently, Vim script only supports the first kind. I think it would be useful to support the second one too.

[brammool]

I think we can leave out the equal sign, just use backticks around the expression.
We can also say that two consecutive backticks stand for one, so that you can still use them without being evaluated.

It looks useful, but before implementing this we should have a discussion about alternatives.
Will this method always work? Is there a better way?

[lacygoill]

I think we can leave out the equal sign, just use backticks around the expression.

We could but:

• we would not be able to use simple backticks to later support shell commands (although =system() would be a workaround)
• it would be inconsistent with how backticks work on the command-line (e.g. after :edit)

Although, I admit it would be more convenient (because in practice, we would probably need to evaluate a Vim script expression more often than a shell command).

We can also say that two consecutive backticks stand for one, so that you can still use them without being evaluated.

That looks a bit weird. Usually, when a character has a special meaning and we want it to be literal, we escape it (usually with a backslash).

Will this method always work? Is there a better way?

FWIW, UltiSnips (which is one of the most popular snippets plugin) uses a similar syntax to let the user evaluate code inside snipppets.

Inside a snippet, this is shell code:

`shell command`

This is Vim script code:

`!v Vim script expression`

This is Python code:

`!p python statement`

And this is a literal backtick:

\`

I think other snippets plugins use a similar kind of syntax.

[LemonBoy]

The use of backticks a-la JS templated strings was already discussed (and implemented in #4634), expanding the logic to work in heredocs may help solve this problem.

[brammool]

> I think we can leave out the equal sign, just use backticks around the expression. We could but: - we would not be able to use simple backticks to later support shell commands (although `=system()` would be a workaround)

Using shell commands would be (or should be) rare. and you can use `system(cmd)`.

- it would be inconsistent with how backticks work on the command-line (e.g. after `:edit`)

True. But then backtick without equal sign would mean using a shell command. Which is really slow and has escaping problems.

Although, I admit it would be more convenient (because in practice, we would probably need to evaluate a Vim script expression more often than a shell command).

Right.

> We can also say that two consecutive backticks stand for one, so that you can still use them without being evaluated. That looks a bit weird. Usually, when a character has a special meaning and we want it to be literal, we escape it (usually with a backslash).

The problem with using a backslash is that this makes backslash a special character, then you would need to double a backslash to get one. Otherwise, how would you get a literal \` ? And backslashes are used often, while if we only make backtick special it's a lot simpler. Compare this with single quoted strings, where the only the single quote is special, needs to be doubled to get one. Heredocs are similar, in that nearly everything is taken literally.

…

-- The problem with political jokes is that they get elected. /// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\ /// \\\ \\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[lacygoill]

Fixed by patch 8.2.4770

[vim-ml]

Hi,

On Sat, Apr 9, 2022 at 1:20 AM lacygoill ***@***.***> wrote: *Is your feature request about something that is currently impossible or hard to do? Please describe the problem.* It is not easy to include the value of a Vim script expression inside a heredoc: vim9script var lines =<< trim END echo $HOME END echo lines ['echo $HOME'] If we want $HOME to be evaluated, we need an extra call to a combination of map() and substitute(): vim9script var lines =<< trim END echo $HOME END lines->map((_, line) => line->substitute('$HOME', $HOME, '')) echo lines [echo /home/user] Which makes the code less readable, and is probably not always reliable in the general case (i.e. substitute() might replace tokens which we might want to be left alone). *Describe the solution you'd like* Extend the backtick expansion syntax which supports Vim script expressions (the one documented at :help E1083 <https://vimhelp.org/editing.txt.html#E1083>) to heredocs: :e `=tempname()` ^^ ^ With this feature, the previous snippet could be simplified into this: vim9script var lines =<< trim eval END echo `=$HOME` END echo lines Notice the presence of the new eval argument on the first line of the assigment: var lines =<< trim eval END ^--^ It would be necessary to avoid breaking existing scripts. With eval, Vim would look for Vim script expressions to evaluate. *Describe alternatives you've considered* An alternative is to continue using an extra call to a combination of map() and substitute(), which – again – is less readable and less reliable than the proposed syntax. *Additional context* In bash, there are 2 kinds of heredocs. The ones which do not try to expand any parameter: cat <<'EOF' echo $HOME EOF echo $HOME And the ones which do: cat <<EOF echo $HOME EOF echo /home/user The difference between the 2 depends on whether the end word is quoted. Currently, Vim script only supports the first kind. I think it would be useful to support the second one too.

I have created PR #10138 with the support for this. I have tested this only with the legacy script. Can you try this out and let me know if you see any issues? Thanks, Yegappan