Here documents don't work

[dusanbrankov]

Bashly Version

Latest Ruby Gem

Description

Hi there,

I just recently started using Bashly and it's been a really great experience so far. However, I noticed that adding a simple here document causes a syntax error (unexpected end of file).

The problem is that the closing EOF token must not have any whitespace before it, but the code is placed inside a function, so naturally everything inside is indented.

See Reproduction Steps below.

For now, I need to run sed -r 's/\s+(EOF)/\1/' app before running the generated script. However, it would be great if the framework could handle this, so that no whitespace is added before each end token.

Note that I also tried opening the here document with <<- instead of <<, but that only works with tabs, not spaces.

More information about this error:

• SC1040 – ShellCheck Wiki
• SC1039 – ShellCheck Wiki

Best,
Dušan

Contents of bashly.yml

No response

Reproduction Steps

Here's an example:

# some_command.sh

cat << EOF
some
text
EOF

With this code I get:

$ bashly generate && ./app
./gp: line 604: warning: here-document at line 268 delimited by end-of-file (wanted `EOF')
./gp: line 605: syntax error: unexpected end of file

The generated output:

some_command() {
  cat << EOF
  some
  text
  EOF
}

This doesn't work because the closing EOF is also indented.

Correct would be:

some_command() {
  cat << EOF
  some
  text
EOF
}

Actual Behavior

No response

Expected Behavior

No response

[DannyBen]

Bashly cares about generating properly indented, human readable code. Heredoc had to be sacrificed.

This was discussed in a few other issues.

There are a couple of possible workarounds in the Examples section.

I am open to ideas on how to better handle this, assuming there is an elegant way.

[dusanbrankov]

Thanks for the quick reply and the links.

I was looking for similar issues but couldn't find anything directly related to Bashly. Sorry for the duplication.

Maybe it's just me, but I find both existing solutions rather clumsy, especially the one with BASHLY_TAB_INDENT, which requires the use of Unicode-specific spaces instead of "normal" ones.

In the meantime, I've written a script that allows me to write here documents in a more natural way:

cat <<EOF
line one
line two
EOF

The following variants also work:

cat << EOF
line one
line two
EOF

cat << EOF > output.txt
hello
there
EOF

If I want to preserve whitespace, I can use <<- (probably it should be the other way around):

cat <<-EOF
line one
    line two (indented)
EOF

And here's the script to make it work:

#!/usr/bin/env bash

executable="$(awk 'NR == 1 {sub(/name:\s*/, ""); print; exit}' src/bashly.yml)"

bashly generate

# Remove leading spaces from end token
sed -i -r 's/^\s+(EOF)/\1/' "$executable"

# Remove leading spaces from docstring (not tabs)
while read -r num line; do
    if [[ -n $line ]]; then
        sed -i -r "$num s/^ +(.+)/\1/" "$executable"
    fi
done <<EOF
$(awk -v lines="$(wc -l "$executable" | awk '{print $1}')" '{\
    if (/^EOF/ || /<<- *EOF/) exit; \
    if (/<< *EOF/) lines = NR; \
    if (NR > lines) print NR, $0; \
}' "$executable")
EOF

The limitation I have with this is that I have to use EOF, but this could be solved with an environment variable, e.g. HEREDOC_TOKEN=CUSTOM.

Also, I haven't thought about \EOF and 'EOF' yet. There are probably other possible forms that I haven't covered, but I think this is a good starting point.

[DannyBen]

Sorry for the duplication.

It's ok.

Maybe it's just me, but I find both existing solutions rather clumsy

Not just you. I agree the workarounds are clumsy. But so is the implementation of heredoc in bash itself.
In other languages, there is a way to keep the EOF marker indented.

I am sure there is a way to make bashly more heredoc-friendly, I just haven't found it yet... :)

[dusanbrankov]

Not just you. I agree the workarounds are clumsy. But so is the implementation of heredoc in bash itself. In other languages, there is a way to keep the EOF marker indented.

Totally agree with you. Bash is just bad in so many ways :)

I will close this for now as this is a known issue with existing workarounds.

Thanks again for your help.

Best,
Dušan

[DannyBen]

Sure thing. Since this bothers more than a handful of people, I will see if there is anything to be done to better support this.

[DannyBen]

I have made heredoc work naturally, without the need to do anything (#496). bashly will no longer apply additional indentation to the strings inside a heredoc block.

This is available in the edge version, if anyone wishes to test it.