wordwrap filter ignores existing newlines

[dwt]

Hi there,

I've noticed that when using this filter to (for example) ensuring an email message is always wrapped to 72 characters (where possible) to ensure a nice display of it as often as possible, we noticed that wordwrap would not use existing newlines in the message as clue but would instead insert newlines strictly every 72 characters.

This is our workaround

## Workaround bug in do_wordwrap where it disregards existing linebreaks 
## when wrapping text

from jinja2.filters import environmentfilter
import re
@environmentfilter
def do_wordwrap(environment, s, width=79, break_long_words=True):
    """
    Return a copy of the string passed to the filter wrapped after
    ``79`` characters.  You can override this default using the first
    parameter.  If you set the second parameter to `false` Jinja will not
    split words apart if they are longer than `width`.
    """
    import textwrap
    accumulator = []
    # Workaround: pre-split the string
    for component in re.split(r"\r?\n", s):
        # textwrap will eat empty strings for breakfirst. Therefore we route them around it.
        if len(component) is 0:
            accumulator.append(component)
            continue
        accumulator.extend(
            textwrap.wrap(component, width=width, expand_tabs=False,
                replace_whitespace=False,
                break_long_words=break_long_words)
        )
    return environment.newline_sequence.join(accumulator)

Not nice and complete but works for us.

I'd like this (or something similar) to be included in jinja2 proper, as it makes the wordrwap filter much more usefull.

What do you think?

[untitaker]

I don't think this is a bug. Wordwrap just seems to assume that your text is not wrapped yet.

[dwt]

I don't think this is a bug. Wordwrap just seems to assume that your text is not wrapped yet.

Well, obviously. But it makes it really hard to use in an environment where you need the wrapping but the text you want to wrap already contains Some form of formatting (to separate Paragraphs for example).

[rbu]

Any news on this one? We have plain text emails where we need to ensure paragraphs are wrapped at the end, but want lines within the paragraph to be wrapped automatically. This seems to be a common use case. In fact, this comment box I am typing this message in does the same. These lines are wrapped automatically, but when I hit double enter

a new paragraph starts.

[untitaker]

Couldn't you use the stdlib email module for this?

[rbu]

At which point would you insert this?
We're using Jinja2 to generate and localize mails and after compiling the template, hand the resulting string over to pyramid_mailer/repoze_sendmail.

A plain-text email template looks something like this:

{%- filter wordwrap(width=72, break_long_words=False) -%}
{% block greeting -%}
{% trans full_name = _(user.full_name) %}Hello {{ full_name }},{% endtrans %}
{% endblock -%}

{% block message_intro %}
{% endblock -%}

{% trans -%}
This may be a very long text in another language, depending on what a translator put into the gettext localization. It may even have its own paragraphs.
{%- endtrans %}

...

[fossilet]

I also encountered this problem and a Google search led me here. Using the patch from @dwt works for me. I also vote for this change to be incorporated in the code base.

[p2]

Same here, would be great to incorporate the fix into master!

Note that wrapstring disappeared in the patch

[very-emmazing]

I submitted a pull requests that works around this somewhat unexpected behaviour of textwrap.wrap() when dealing with multiple paragraphs.

wrapstring and empty lines are preserved.

We're using this functionality for generating email bodies as part of another application and want to wrap input at 80 columns.

[t-makaro]

I'm encountering this issue as well in my nbconvert template. It'd be super nice to have #766 merged since if I have to write my own filter, I believe that I'll have to write a custom exporter for nbconvert, which for ease of use I'm trying to avoid.