Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

text/tabwriter: add cell width function #12073

Open
jszwedko opened this issue Aug 7, 2015 · 12 comments
Open

text/tabwriter: add cell width function #12073

jszwedko opened this issue Aug 7, 2015 · 12 comments
Labels
NeedsDecision
Milestone

Comments

@jszwedko
Copy link
Contributor

@jszwedko jszwedko commented Aug 7, 2015

What version of Go are you using (go version)?

1.4.2

What operating system and processor architecture are you using?

Linux / AMD64

What did you do?

package main

import (
        "fmt"
        "os"
        "text/tabwriter"
)

func main() {
        t := tabwriter.NewWriter(os.Stdout, 0, 8, 0, '\t', 0)
        fmt.Fprintf(t, "\x1b[31mfoo\x1b[0m\tbar\n")
        fmt.Fprintf(t, "foo\tbar\n")
        t.Flush()
}

What did you expect to see?

foo     bar
foo     bar

Where the first foo is red.

What did you see instead?

foo     bar
foo             bar

Where the first foo is red due to non-printable ANSI escape sequences being included in cell width calculations.

Proposal:

I realize that printing to the terminal may not be text/tabwriters intended purpose, but I think it would be nice to be able to configure how the width of a cell is calculated as I can imagine further cases (similar to the special casing of HTML tags and entities currently) that would also benefit from this.

Suggestion: Exposing an additional field on the tabwriter.Writer struct so as to maintain the function signature of Init and NewWriter:

type Writer struct {
    CellWidth func([]byte) int
}

If this function was nil, the existing width calculation could be used, but otherwise this function could be called with the contents of the cell.

I can take a stab at the implementation of this, but I first wanted to see if such a change would be welcome.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Aug 8, 2015

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Aug 8, 2015
@griesemer
Copy link
Contributor

@griesemer griesemer commented Aug 10, 2015

@jszwedko I think this is a reasonable suggestion, and in retrospect perhaps this would have been a better way to customize cell measurement than with the specific flags we have now. That said, I'm not sure how easy it would be to make this work given that part of the current implementation is using a state-machine like mechanism (but I haven't looked into it in detail).

Specifically, my concern is how such a function would interact or possibly interfere with what we have already.

Alternatively, an additional flag could be provided that would exclude escaped sequences of characters from the width computation. Or perhaps another escape character. Both these might be more in the current spirit of things even though perhaps less flexible than the function.

If you want to give it a shot and come up with a concrete implementation, please feel free to go ahead. I'm happy to review but will push back if the result doesn't fit nicely with what we have.

@jszwedko
Copy link
Contributor Author

@jszwedko jszwedko commented Aug 11, 2015

@griesemer yeah, looking more closely at the way the width is currently calculated does lead me to believe that it would be difficult to integrate a function like I suggested. I like the idea of a flag to exclude escaped characters, but it does feel like it should be a separate escape sequence to allow both features to be used concurrently. I'll mock that up and see what it looks like.

Thanks for the feedback!

@mpvl
Copy link
Contributor

@mpvl mpvl commented Aug 17, 2015

This is somewhat related to the request to render CJK characters as two characters. In general, there are other classes of runes for which one would need to use alternative widths: fullwidth, modifiers, Jamo V+T, etc. If we make a change to the width handling, we should take these other cases in to account and allow supporting it. We could limit ourselves to determining width on a rune-by-rune basis.

The column width of runes could look something like this (based on http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c, but slightly different):

  • 0: Cc control characters
  • 0: Non-spacing and enclosing combining characters (Mn/Me).
  • 1: SOFT HYPHEN (U+00AD) (Cf)
  • 0: Other format characters (Cf)
  • 0: Hangul Jamo V+T medial vowels and final consonants (U+1160-U+11FF)
  • 2: Spacing characters of type EastAsianWide (W) or EastAsianFullwidth (F) as defined in Unicode TR#11 (see also golang.org/x/text/width and width.Kind).
  • 1: All remaining characters, including all printable, Unicode control characters, and width.EastAsianAmbiguous.

@adg adg added Proposal and removed Proposal labels Sep 25, 2015
@adg adg changed the title proposal: text/tabwriter cell width function text/tabwriter: add cell width function Sep 30, 2015
@adg adg removed the Proposal label Sep 30, 2015
@WeiZhang555
Copy link

@WeiZhang555 WeiZhang555 commented Jan 18, 2016

Hello, everyone, I meet the same problem of printing CJK words, so I have to write a custom tabwriter (WeiZhang555/tabwriter).
I read code of http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c before implementation, but I use a very similar Golang git repository "github.com/moznion/go-unicode-east-asian-width" instead and only changes little code: here https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L408-L413 and here https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L388-L404. You can regard this as a POC.

so the question is:

  1. Do you think this is OK or not? I mean changing the default width calculation method?
  2. The proposal mentioned custom CellWidth calculation method, I think it can also give me some help, why you say this can't be implemented ?

@mpvl
Copy link
Contributor

@mpvl mpvl commented Jan 18, 2016

Changing the default is not an option, imo. CJK scripts are not the only
one that vary in width. In fact, even the width ratios varies for CJK
depending on the font used. For someone who's primary development
environment is C, J, or K the 2:1 ratio is very likely. For users of
Latin-oriented fixed-width fonts, for example, it is not. (I believe it is
5/3:1.) Furthermore, there are also characters that should be mapped to
zero width and characters for which it is unclear to which width they
should be mapped in general. Overall it is very hard, if not impossible, to
come up with a single mapping that works across the board.

What I could imagine, though, is allowing tabwriter to have an optional
interface that maps the length of an element or single character. It seems
hard to extend the current NewWriter function to add this. I can imagine,
though, adding a New function that takes an option argument:

New(w io.Writer, opts ...Option) *Writer

type Option ...

func Padding(n int) Option
func TabWidth(width ...int) Option // would allow different widths per
column
func MinWidth(width ...int) Option
func PadChar(r rune) Option

func WidthFunc(func(cell []byte) int) Option

Example:

tabwriter.New(w, tabwriter.TabWidth(30),
tabwriter.WidthFunc(width.FixedWidthEastAsian))

The flags are, unfortunately, not typed, otherwise they could be options.
This is the biggest problem with adopting this API, I think. Haven't given
it much thought, though. In the worst case, this package could be copied
into the text repo, but that would be lame. I rather not do that.

Something like that. That is quite an addition to the current package, even
though it is only a new API wrapper, so that probably requires a proposal.
WidthFunc is defined on the entire cell, instead of per rune, to be able to
handle contextual sizes (such as Hangul rendering for decomposed Jamo in
Korean).

Note that the golang.org/x/text/width package also has support for
East-Asian width. This package could provide implementations of the
interface for tabwriter. All the data is there.
Similarly a package for Arabic Shaping could provide approximate widths for
Arabic (don't actually know if there is such a thing as fixed-width Arabic).

On Mon, Jan 18, 2016 at 7:34 AM, zhangwei_cs notifications@github.com
wrote:

Hello, everyone, I meet the same problem of printing CJK words, so I have
to write a custom tabwriter (WeiZhang555/tabwriter
https://github.com/WeiZhang555/tabwriter).
It uses a very similar Golang git repository "
github.com/moznion/go-unicode-east-asian-width" and only changes little
code: here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L408-L413
and here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L388-L404.
You can regard this as a POC.

so the question is:

  1. Do you think this is OK or not? I mean changing the default width
    calculation method?
  2. The proposal mentioned custom CellWidth calculation method, I think it
    can also give me some help, why you say this can't be implemented ?


Reply to this email directly or view it on GitHub
#12073 (comment).

@WeiZhang555
Copy link

@WeiZhang555 WeiZhang555 commented Jan 18, 2016

Sounds really complicated, or even impossible! :-(

@mpvl
Copy link
Contributor

@mpvl mpvl commented Jan 18, 2016

Of course the alternative is to add a WriteFunc member to Writer. This is
somewhat messy and ties the implementation to a single way of doing things,
but may be best, given the flag issue.

On Mon, Jan 18, 2016 at 10:25 AM, mpvl@golang.org wrote:

Changing the default is not an option, imo. CJK scripts are not the only
one that vary in width. In fact, even the width ratios varies for CJK
depending on the font used. For someone who's primary development
environment is C, J, or K the 2:1 ratio is very likely. For users of
Latin-oriented fixed-width fonts, for example, it is not. (I believe it is
5/3:1.) Furthermore, there are also characters that should be mapped to
zero width and characters for which it is unclear to which width they
should be mapped in general. Overall it is very hard, if not impossible, to
come up with a single mapping that works across the board.

What I could imagine, though, is allowing tabwriter to have an optional
interface that maps the length of an element or single character. It seems
hard to extend the current NewWriter function to add this. I can imagine,
though, adding a New function that takes an option argument:

New(w io.Writer, opts ...Option) *Writer

type Option ...

func Padding(n int) Option
func TabWidth(width ...int) Option // would allow different widths per
column
func MinWidth(width ...int) Option
func PadChar(r rune) Option

func WidthFunc(func(cell []byte) int) Option

Example:

tabwriter.New(w, tabwriter.TabWidth(30),
tabwriter.WidthFunc(width.FixedWidthEastAsian))

The flags are, unfortunately, not typed, otherwise they could be options.
This is the biggest problem with adopting this API, I think. Haven't given
it much thought, though. In the worst case, this package could be copied
into the text repo, but that would be lame. I rather not do that.

Something like that. That is quite an addition to the current package,
even though it is only a new API wrapper, so that probably requires a
proposal. WidthFunc is defined on the entire cell, instead of per rune, to
be able to handle contextual sizes (such as Hangul rendering for decomposed
Jamo in Korean).

Note that the golang.org/x/text/width package also has support for
East-Asian width. This package could provide implementations of the
interface for tabwriter. All the data is there.
Similarly a package for Arabic Shaping could provide approximate widths
for Arabic (don't actually know if there is such a thing as fixed-width
Arabic).

On Mon, Jan 18, 2016 at 7:34 AM, zhangwei_cs notifications@github.com
wrote:

Hello, everyone, I meet the same problem of printing CJK words, so I have
to write a custom tabwriter (WeiZhang555/tabwriter
https://github.com/WeiZhang555/tabwriter).
It uses a very similar Golang git repository "
github.com/moznion/go-unicode-east-asian-width" and only changes little
code: here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L408-L413
and here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L388-L404.
You can regard this as a POC.

so the question is:

  1. Do you think this is OK or not? I mean changing the default width
    calculation method?
  2. The proposal mentioned custom CellWidth calculation method, I think it
    can also give me some help, why you say this can't be implemented ?


Reply to this email directly or view it on GitHub
#12073 (comment).

@mpvl
Copy link
Contributor

@mpvl mpvl commented Jan 18, 2016

The implementation is really not hard. The biggest problem is coming up
with an acceptable API extension for tabwriter.

The second issue is to decide how the mapping looks like for fixed-width
CJK, also taking into account modifiers, etc. Luckily people have thought
about this and there are some good definitions that are easy to implement.
The width package contains all (or almost all) data that is needed to
implement this.

The problem with any fixed mapping, though, is that they don't work in all
situations. That's why it shouldn't be tabwriter implementing this.
Otherwise it is not hard. It just needs to be coordinated with some
tabwriter API extension.

On Mon, Jan 18, 2016 at 10:48 AM, zhangwei_cs notifications@github.com
wrote:

Sounds really complicated, or even impossible! :-(


Reply to this email directly or view it on GitHub
#12073 (comment).

@WeiZhang555
Copy link

@WeiZhang555 WeiZhang555 commented Jan 23, 2016

@griesemer
I think this is the CL you asked me to send: https://go-review.googlesource.com/18891, sorry that I'm not familiar with the review system and it took me some time.

And this is a very preliminary implementation, and I understand that it's not good enough, but at least it shows my thought. Thank you!

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 23, 2016

CL https://golang.org/cl/18891 mentions this issue.

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 20, 2019

Change https://golang.org/cl/202257 mentions this issue: text/tabwriter: add ANSI Graphics Rendition format

@ALTree ALTree added the NeedsDecision label Dec 4, 2019
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 7, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 7, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 7, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 8, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 8, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 8, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 8, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 8, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 8, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 10, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 10, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 10, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 11, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 12, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 12, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 12, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 12, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 13, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 13, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
debarshiray pushed a commit to debarshiray/toolbox that referenced this issue May 13, 2020
Go's text/tabwriter package isn't aware of the ANSI Select Graphic
Rendition (or SGR) escape sequences to change the terminal's
foreground colour [1,2]. Using the sequences disrupt the alignment of
the columns in the tabbed output.

Therefore, unlike the existing POSIX shell implementation, running
containers aren't marked in green in the Go version.

[1] https://en.wikipedia.org/wiki/ANSI_escape_code
[2] golang/go#12073

containers#318
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsDecision
Projects
None yet
Development

No branches or pull requests

9 participants