Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wordwrap filter ignores existing newlines #175

Closed
dwt opened this issue Feb 5, 2013 · 9 comments · Fixed by #766
Closed

wordwrap filter ignores existing newlines #175

dwt opened this issue Feb 5, 2013 · 9 comments · Fixed by #766
Milestone

Comments

@dwt
Copy link

@dwt dwt commented Feb 5, 2013

Hi there,

I've noticed that when using this filter to (for example) ensuring an email message is always wrapped to 72 characters (where possible) to ensure a nice display of it as often as possible, we noticed that wordwrap would not use existing newlines in the message as clue but would instead insert newlines strictly every 72 characters.

This is our workaround

## Workaround bug in do_wordwrap where it disregards existing linebreaks 
## when wrapping text

from jinja2.filters import environmentfilter
import re
@environmentfilter
def do_wordwrap(environment, s, width=79, break_long_words=True):
    """
    Return a copy of the string passed to the filter wrapped after
    ``79`` characters.  You can override this default using the first
    parameter.  If you set the second parameter to `false` Jinja will not
    split words apart if they are longer than `width`.
    """
    import textwrap
    accumulator = []
    # Workaround: pre-split the string
    for component in re.split(r"\r?\n", s):
        # textwrap will eat empty strings for breakfirst. Therefore we route them around it.
        if len(component) is 0:
            accumulator.append(component)
            continue
        accumulator.extend(
            textwrap.wrap(component, width=width, expand_tabs=False,
                replace_whitespace=False,
                break_long_words=break_long_words)
        )
    return environment.newline_sequence.join(accumulator)

Not nice and complete but works for us.

I'd like this (or something similar) to be included in jinja2 proper, as it makes the wordrwap filter much more usefull.

What do you think?

@untitaker

This comment has been minimized.

Copy link
Member

@untitaker untitaker commented Feb 5, 2013

I don't think this is a bug. Wordwrap just seems to assume that your text is not wrapped yet.

@dwt

This comment has been minimized.

Copy link
Author

@dwt dwt commented Feb 6, 2013

I don't think this is a bug. Wordwrap just seems to assume that your text is not wrapped yet.

Well, obviously. But it makes it really hard to use in an environment where you need the wrapping but the text you want to wrap already contains Some form of formatting (to separate Paragraphs for example).

@rbu

This comment has been minimized.

Copy link

@rbu rbu commented Feb 10, 2014

Any news on this one? We have plain text emails where we need to ensure paragraphs are wrapped at the end, but want lines within the paragraph to be wrapped automatically. This seems to be a common use case. In fact, this comment box I am typing this message in does the same. These lines are wrapped automatically, but when I hit double enter

a new paragraph starts.

@untitaker

This comment has been minimized.

Copy link
Member

@untitaker untitaker commented Feb 10, 2014

Couldn't you use the stdlib email module for this?

@rbu

This comment has been minimized.

Copy link

@rbu rbu commented Feb 10, 2014

At which point would you insert this?
We're using Jinja2 to generate and localize mails and after compiling the template, hand the resulting string over to pyramid_mailer/repoze_sendmail.

A plain-text email template looks something like this:

{%- filter wordwrap(width=72, break_long_words=False) -%}
{% block greeting -%}
{% trans full_name = _(user.full_name) %}Hello {{ full_name }},{% endtrans %}
{% endblock -%}

{% block message_intro %}
{% endblock -%}

{% trans -%}
This may be a very long text in another language, depending on what a translator put into the gettext localization. It may even have its own paragraphs.
{%- endtrans %}

...
fossilet added a commit to fossilet/jinja2 that referenced this issue Apr 4, 2014
@fossilet

This comment was marked as off-topic.

Copy link

@fossilet fossilet commented Apr 4, 2014

I also encountered this problem and a Google search led me here. Using the patch from @dwt works for me. I also vote for this change to be incorporated in the code base.

@p2

This comment was marked as off-topic.

Copy link

@p2 p2 commented Sep 23, 2014

Same here, would be great to incorporate the fix into master!

Note that wrapstring disappeared in the patch

fheinle pushed a commit to fheinle/jinja that referenced this issue Sep 7, 2017
textwrap.wrap() from stdlib has unexpected behaviour where when
wrapping multiple paragraphs it will not consider existing newlines.

I.e. when your first paragraph ends on col 75, the next paragraph
will be wrapped on col 5 already.

This patch is wrapping each line individually and combining it back
together.

Fixes pallets#175.
@fheinle

This comment has been minimized.

Copy link

@fheinle fheinle commented Sep 7, 2017

I submitted a pull requests that works around this somewhat unexpected behaviour of textwrap.wrap() when dealing with multiple paragraphs.

wrapstring and empty lines are preserved.

We're using this functionality for generating email bodies as part of another application and want to wrap input at 80 columns.

@t-makaro

This comment has been minimized.

Copy link

@t-makaro t-makaro commented May 15, 2018

I'm encountering this issue as well in my nbconvert template. It'd be super nice to have #766 merged since if I have to write my own filter, I believe that I'll have to write a custom exporter for nbconvert, which for ease of use I'm trying to avoid.

@davidism davidism added i18n and removed i18n labels Oct 19, 2019
@davidism davidism added this to the 2.11.0 milestone Nov 1, 2019
@davidism davidism closed this in #766 Nov 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

8 participants
You can’t perform that action at this time.