Add unicode-bidi to support RTL languages

[amitlevy21]

Is your feature request related to a problem? Please describe.
Right to Left (RTL) languages like Hebrew and Arabic don't work well with go-pretty table functionality.
The order of columns doesn't map well, and the headers spacing is also not formatted.

This picture is taken from Konsole, a terminal that supports unicode-bidi

Describe the solution you'd like
I'd like the columns to show nicely as they do for English.
There's built in support for unicode bidi in go: https://pkg.go.dev/golang.org/x/text/unicode/bidi

Describe alternatives you've considered
I could try to translate them from Hebrew to English, but it would result in loss of data in case the translation is not correct, so it won't work for me.

Additional context
Add any other context or screenshots about the feature request here.

Thank you for this great tool and your time.

[jedib0t]

Can you share a sample program with dummy data, and a hand-crafted "expected" output? I've got no clue about Hebrew/Arabic, and would like to know what the input will look like and what the output should look like before I can say if this is something that this library can/should support. I'd prefer text answer that I can run myself, instead of an image like above.

Thanks!

[amitlevy21]

Thanks for getting back so quickly!

I've made a repo with example code and expected results. While crafting the expected output, Iv'e encounter issues where I couldn't format it to the desired results without wrapping some strings with []. So I test it in the code and found out I almost get the desired result by wrapping each header string in the first row and any string with []
https://github.com/amitlevy21/go-pretty-bidi-example

In main.go you would find some comments indicating the changes i need to make to adapt RTL to show properly. Still, in with_wrapping_[].txt you would see some spaces not align properly, but It's good enough for me.

Do notice this MUST be tested in a terminal that supports bidi, like Konsole, Gnome terminal and more listed here https://unix.stackexchange.com/questions/100811/are-there-terminal-emulators-that-support-bi-directional-text

Thanks!

[jedib0t]

I rewrote some of the code for simplicity to my benefit as follows:

• https://go.dev/play/p/hdm8CUOvJZe (old)
• https://go.dev/play/p/Lj9DVSQ2fsM

Output:

+---+------------+--------+----------+-----------+
| # | [תאריך]    | [סכום] | [מחלקה]  | [תגים]    |
+---+------------+--------+----------+-----------+
| 0 | 2021-03-18 |      5 | [מחלקה1] | [תג1 תג2] |
| 1 | 2021-04-19 |      5 | [מחלקה1] | [תג1]     |
| 2 | 2021-05-20 |      5 | [מחלקה2] | [תג1]     |
+---+------------+--------+----------+-----------+
|   | [סהכ]      |     30 |          |           |
+---+------------+--------+----------+-----------+
Wrapped: true

+---+------------+-------+--------+-----------+
| # | תגים       | מחלקה | סכום   | תאריך     |
+---+------------+-------+--------+-----------+
| 0 | 2021-03-18 |     5 | מחלקה1 | [תג1 תג2] |
| 1 | 2021-04-19 |     5 | מחלקה1 | [תג1]     |
| 2 | 2021-05-20 |     5 | מחלקה2 | [תג1]     |
+---+------------+-------+--------+-----------+
|   | סהכ        |    30 |        |           |
+---+------------+-------+--------+-----------+
Wrapped: false

So if I understand it right, the proper output is the first one but without the brackets. Correct?

[amitlevy21]

Yes, exactly

[jedib0t]

Okay. From my testing, the direction is being auto-determined by some golang internals and not by go-pretty. I'll try to see if there is some way to control this without letting it auto-decide based on content so that I can expose the same control to users of go-pretty.

But here is a workaround using the table styling with not-so-good-looking-output. Until I find a way forward for this issue at least. 😄

Use this before calling t.Render():

	t.Style().Box = table.BoxStyle{
		BottomLeft:       "[",
		BottomRight:      "]",
		BottomSeparator:  "][",
		EmptySeparator:   "  ",
		Left:             "[",
		LeftSeparator:    "[",
		MiddleHorizontal: "-",
		MiddleSeparator:  "][",
		MiddleVertical:   "][",
		PaddingLeft:      " ",
		PaddingRight:     " ",
		PageSeparator:    "\n",
		Right:            "]",
		RightSeparator:   "]",
		TopLeft:          "[",
		TopRight:         "]",
		TopSeparator:     "][",
		UnfinishedRow:    " ~",
	}

Output:

[---][------------][--------][----------][-----------]
[ # ][ [תאריך]    ][ [סכום] ][ [מחלקה]  ][ [תגים]    ]
[---][------------][--------][----------][-----------]
[ 0 ][ 2021-03-18 ][      5 ][ [מחלקה1] ][ [תג1 תג2] ]
[ 1 ][ 2021-04-19 ][      5 ][ [מחלקה1] ][ [תג1]     ]
[ 2 ][ 2021-05-20 ][      5 ][ [מחלקה2] ][ [תג1]     ]
[---][------------][--------][----------][-----------]
[   ][ [סהכ]      ][     30 ][          ][           ]
[---][------------][--------][----------][-----------]
Wrapped: true

[---][------------][------][--------][---------]
[ # ][ תאריך      ][ סכום ][ מחלקה  ][ תגים    ]
[---][------------][------][--------][---------]
[ 0 ][ 2021-03-18 ][    5 ][ מחלקה1 ][ תג1 תג2 ]
[ 1 ][ 2021-04-19 ][    5 ][ מחלקה1 ][ תג1     ]
[ 2 ][ 2021-05-20 ][    5 ][ מחלקה2 ][ תג1     ]
[---][------------][------][--------][---------]
[   ][ סהכ        ][   30 ][        ][         ]
[---][------------][------][--------][---------]
Wrapped: false

[amitlevy21]

Thanks for looking into this! the workaround are totally fine by me :)

[jedib0t]

Hey @amitlevy21 I've added support for forcing the text direction for tables with BiDi content: #230

Can you try your code without wrapping, with the following directive:

    t.Style().Format.Direction = text.LeftToRight

Once you verify functionality with the latest content from the main branch, I'll cut a tag for you.

[amitlevy21]

Thanks @jedib0t ! It's much better, there is a small indentation issue which I'm not sure if its specific to bidi or not.
If the slice of tags is empty, it can cause this wrong spacing in the last column:

func NewTestExpenses() *Expenses {
	return &Expenses{Classified: []*Expense{
		{
			Date:   UTCDate(2021, 03, 18),
			Amount: 5.0,
			Class:  "הי         י מחלקה1",
			Tags:   []Tag{"תג1", "תג2"},
		},
		{
			Date:   UTCDate(2021, 04, 19),
			Amount: 5.0,
			Class:  "מחלקה1",
			Tags:   []Tag{"תג1"},
		},
		{
			Date:   UTCDate(2021, 05, 20),
			Amount: 5.0,
			Class:  "מחלקה נסיון אחת שתיים",
			Tags:   []Tag{},
		},
		{
			Date:   UTCDate(2021, 05, 20),
			Amount: 5.0,
			Class:  "מחלקה נסיון אחת שתיים שלוש2",
			Tags:   []Tag{},
		},
	}}
}

[jedib0t]

This may be an issue with the terminal emulator you are using, or the font/typeface. Here is how it looks on my system:

Code used with table@v6.3.9

package main

import (
	"fmt"
	"strings"
	"time"

	"github.com/jedib0t/go-pretty/v6/table"
	"github.com/jedib0t/go-pretty/v6/text"
)

var (
	wrapped = false
)

type Tag = string

type Expense struct {
	Date   time.Time
	Amount float64
	Class  string
	Tags   []Tag
}

func UTCDate(year int, month time.Month, day int) time.Time {
	timeZone, _ := time.LoadLocation("UTC")
	return time.Date(year, month, day, 0, 0, 0, 0, timeZone)
}

func displayExpenses(expenses []*Expense) {
	t := table.NewWriter()
	t.AppendHeader(generateHeader())
	for i, e := range expenses {
		dateWithoutTime := strings.Split(e.Date.String(), " ")[0]
		eClass := processBiDi(e.Class)
		eTags := processBiDi(strings.Join(e.Tags, " "))
		t.AppendRow(table.Row{i, dateWithoutTime, e.Amount, eClass, eTags})
	}
	t.AppendFooter(generateFooter())
	t.SetCaption("Wrapped: %v", wrapped)
	t.Style().Format.Direction = text.LeftToRight

	fmt.Printf("%s\n\n", t.Render())
}

func generateFooter() table.Row {
	row := table.Row{"", "סהכ", 30}
	row[1] = processBiDi(fmt.Sprint(row[1]))
	return row
}

func generateHeader() table.Row {
	row := table.Row{"#"}
	for _, col := range []string{"תאריך", "סכום", "מחלקה", "תגים"} {
		row = append(row, processBiDi(col))
	}
	return row
}

func processBiDi(str string) string {
	if wrapped {
		return fmt.Sprintf("[%s]", str)
	}
	return str
}

func main() {
	testExpenses := []*Expense{
		{
			Date:   UTCDate(2021, 03, 18),
			Amount: 5.0,
			Class:  "הי         י מחלקה1",
			Tags:   []Tag{"תג1", "תג2"},
		},
		{
			Date:   UTCDate(2021, 04, 19),
			Amount: 5.0,
			Class:  "מחלקה1",
			Tags:   []Tag{"תג1"},
		},
		{
			Date:   UTCDate(2021, 05, 20),
			Amount: 5.0,
			Class:  "מחלקה נסיון אחת שתיים",
			Tags:   []Tag{},
		},
		{
			Date:   UTCDate(2021, 05, 20),
			Amount: 5.0,
			Class:  "מחלקה נסיון אחת שתיים שלוש2",
			Tags:   []Tag{},
		},
	}

	displayExpenses(testExpenses)
}

Here it is with tags appended to table as e.Tags instead of using strings.Join:

Terminal: terminator; Font/typeface: Ubuntu mono

Here it is on Gnome Terminal:

[amitlevy21]

I see, looks good to me! thank you for providing this!