[feature request] autolinebreak for fenced code under skylighting

[jannick0]

Pandoc 2.0.6 / skylighting 0.5.0.1:

It appears that for fenced code blocks with long lines there is no autolinebreak feature (for style pygments, at least).

Example:
pandoc -f markdown -t latex -o long-code-line.pdf long-code-line.md

long-code-line.md:
~~~
   this is a very long long long long long long long long long long long long long line which is not broken
~~~

NB: Options of the LaTeX listings package:

• breaklines (boolean)
• breakatwhitespace (boolean)
• breakautoindent
• linewidth

Apologies if there is a trivial solution to this issue making this feature request obsolete.

[jgm]

I don't think this can be implemented in skylighting itself, because skylighting has no way of knowing how much space each token takes and hence when the end of the line is reached.

If there's some latex magic that can cause auto wrapping inside a custom verbatim environment defined using fancyvrb, then we could use that.

[jannick0]

On the LaTeX side I believe the package fvextra (github) could be a nice candidate doing all the tedious work by directly talking to - or rather through - fancyvrb.

I had a short look at fvextra and it looks like it covers many - if not all - options of listings. One of the main and nasty issues is that a broken line should not have a new code line number. Probably worth while having a look at the other package features, too, to see if they could be used in pandoc.

For the solution only two tiny edits to the tex code are needed:

%% add
\usepackage{fvextra}
...
%% add the breaklines option like so
\begin{Highlighting}[breaklines,numbers=left,,]
...

Please feel free to close this issue as you may find appropriate.

---

@jgm Separately, John, two additional observations:

1. Lexer - Leading tabs in the code section are glued to the first token which I am not sure is the intension. Of course, it does not affect the output, but I would have expected the AST to do the white space book keeping separately. Tabs within the line share their leading friends' destiny. Again, this is a bit far off, I just wanted to share this with you.

Example:

~~~
[tab]token
~~~

gets translated to

\NormalTok{[tab]token}

2. Documentation - I was a bit confused that --wrap did not solve the code line breaking issue. So I would suggest to add a comment to the documentation saying that this option applies to standard text only.

HTH.

[jannick0]

These edits in the preamble would work, too:

\usepackage{fvextra}
\DefineVerbatimEnvironment{Highlighting}{Verbatim}{breaklines,commandchars=\\\{\}}

[georgeraraujo]

These edits in the preamble would work, too:

\usepackage{fvextra}
\DefineVerbatimEnvironment{Highlighting}{Verbatim}{breaklines,commandchars=\\\{\}}

How about prepending to verbatim instead of creating a custom environment?

% single line spacing for code
\usepackage{etoolbox}
\BeforeBeginEnvironment{verbatim}{\def\baselinestretch{1}}

[jannick0]

baselinestretch changes the spacing in everything of the document after its has been triggered. That is not the aim of the proposed feature, namely line beaks of long lines in fenced code blocks. So I can not see how this would help solving the issue.

BTW - my suggestion would amend an already existing 'custom' environment created by the current version of pandoc. The adjustment is tiny, and it fully addresses the issue.

Or am I missing anything here, since I am lightyears away from being a LaTeX expert?

[georgeraraujo]

baselinestretch changes the spacing in everything of the document after its has been triggered. That is not the aim of the proposed feature, namely line beaks of long lines in fenced code blocks. So I can not see how this would help solving the issue.

Sorry, now I see expressed myself poorly. That was just an example of how it is possible to prepend commands to the existing verbatim environment instead of creating a new one...

BTW - my suggestion would amend an already existing 'custom' environment created by the current version of pandoc. The adjustment is tiny, and it fully addresses the issue.

... but pandoc already does it, so your suggestion to amend looks good to me.

[aubertc]

I also think @jannick0 's suggestion perfectly fits the bill. I took the liberty to mention it as an answer to a question asked on StackExchange, cf. https://tex.stackexchange.com/a/412316/34551

[noraj]

Writing a technical report with code blocks or long string is really a nightmare. Even with fvextra and the breakanywhere option I did not succeed to do something about very long string.
When it's code it is easy to split strings but hen you just want to put the output of a command with very long lines you can't do that.

[jgm]

When it's code it is easy to split strings but hen you just want to put the output of a command with very long lines you can't do that.

If you don't care about where they split, you can use a lua filter to add line breaks automatically.

[noraj]

I'm using markdown to PDF conversion through a LaTeX template made by someone else. I'll rather avoid to have to tweak it.

[jgm]

A filter doesn't require changes to tempate.

[noraj]

@jgm I imagined it was something you put in your LaTeX document, but I looked for it an I saw it was an additional option to pass to pandoc. This make an extra file and I know anything about lua lang.

[tarleb]

Pandoc tries to use only packages which are shipped with texlive-latex-recommended in Debian-based distros. The suggested package fvextra is in texlive-latex-extra. I believe that solving linebreaks in code is out of scope for pandoc.

My personal suggestion would be to use minted, e.g. via the minted Lua filter.

[doitian]

My workaround to handle the code blocks without a language is setting the language to text via a lua filter

function CodeBlock(el)
  if #el.classes == 0 then
    el.classes[1] = 'text'
  end
  return el
end

return {{CodeBlock = CodeBlock}}

Save the file and use it via the pandoc command line option: --lua-filter path/to/the/filter.lua.

[tigerhawkvok]

Perhaps I took a hammer to an egg here, but I was having a hard time line breaking code of regex so I needed break-anywhere. I ended up with this:

---
format:
    html:
        code-fold: true
    pdf:
        header-includes:
            - "\\usepackage{fvextra}"
            - "\\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\\\\{\\}, breaklines, breaknonspaceingroup, breakanywhere}"
            - "\\fvset{breaklines}"
            - "\\fvset{breaknonspaceingroup}"
            - "\\fvset{breakanywhere}"
latex-auto-install: true
---