Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTF-8 (rather, any non-ASCII) evidently dropped/ignored #26

Open
kriskowal opened this issue Nov 13, 2017 · 4 comments
Open

UTF-8 (rather, any non-ASCII) evidently dropped/ignored #26

kriskowal opened this issue Nov 13, 2017 · 4 comments

Comments

@kriskowal
Copy link

@kriskowal kriskowal commented Nov 13, 2017

Steps to reproduce:

Parse, for example, πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦

Expected: constituent UTF-8 bytes of the four unicode characters, and the three unicode zero-width join characters, should all be send to Print.

Actual: handler.Print is not called. No other method appears to be called.

package main

import ansiterm "github.com/Azure/go-ansiterm"

func main() {
	parser := ansiterm.CreateParser("Ground", &handler{})
	parser.Parse([]byte("πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦"))
}

type handler struct{}

func (h *handler) Print(b byte) error {
	panic("not implemented")
}

func (h *handler) Execute(b byte) error {
	panic("not implemented")
}

func (h *handler) CUU(int) error {
	panic("not implemented")
}

func (h *handler) CUD(int) error {
	panic("not implemented")
}

func (h *handler) CUF(int) error {
	panic("not implemented")
}

func (h *handler) CUB(int) error {
	panic("not implemented")
}

func (h *handler) CNL(int) error {
	panic("not implemented")
}

func (h *handler) CPL(int) error {
	panic("not implemented")
}

func (h *handler) CHA(int) error {
	panic("not implemented")
}

func (h *handler) VPA(int) error {
	panic("not implemented")
}

func (h *handler) CUP(int, int) error {
	panic("not implemented")
}

func (h *handler) HVP(int, int) error {
	panic("not implemented")
}

func (h *handler) DECTCEM(bool) error {
	panic("not implemented")
}

func (h *handler) DECOM(bool) error {
	panic("not implemented")
}

func (h *handler) DECCOLM(bool) error {
	panic("not implemented")
}

func (h *handler) ED(int) error {
	panic("not implemented")
}

func (h *handler) EL(int) error {
	panic("not implemented")
}

func (h *handler) IL(int) error {
	panic("not implemented")
}

func (h *handler) DL(int) error {
	panic("not implemented")
}

func (h *handler) ICH(int) error {
	panic("not implemented")
}

func (h *handler) DCH(int) error {
	panic("not implemented")
}

func (h *handler) SGR([]int) error {
	panic("not implemented")
}

func (h *handler) SU(int) error {
	panic("not implemented")
}

func (h *handler) SD(int) error {
	panic("not implemented")
}

func (h *handler) DA([]string) error {
	panic("not implemented")
}

func (h *handler) DECSTBM(int, int) error {
	panic("not implemented")
}

func (h *handler) IND() error {
	panic("not implemented")
}

func (h *handler) RI() error {
	panic("not implemented")
}

func (h *handler) Flush() error {
	return nil
}
@ncw
Copy link

@ncw ncw commented Sep 5, 2018

I can confirm this problem. It happens for a single unicode character such as Β£ too.

@ncw
Copy link

@ncw ncw commented Sep 11, 2018

Note also that some chinese characters cause an error to be reported by the parser "New state of 'nil' is invalid", eg replacing the above main with this one

func main() {
	parser := ansiterm.CreateParser("Ground", &handler{})
	n, err := parser.Parse([]byte("η‹ε¦–ε°ηΊ’ε¨˜"))
	log.Print(n, err)
}

Gives

$ go run ansiterm.go 
2018/09/11 21:29:53 3 New state of 'nil' is invalid.
ncw added a commit to rclone/rclone that referenced this issue Oct 6, 2018
 #2501

This bug causes lots of strange behaviour with non-ASCII characters and --progress

Azure/go-ansiterm#26
ncw added a commit to rclone/rclone that referenced this issue Oct 10, 2018
 #2501

This bug causes lots of strange behaviour with non-ASCII characters and --progress

Azure/go-ansiterm#26
ncw added a commit to rclone/rclone that referenced this issue Oct 11, 2018
 #2501

This bug causes lots of strange behaviour with non-ASCII characters and --progress

Azure/go-ansiterm#26
@ivandeex
Copy link

@ivandeex ivandeex commented Nov 16, 2020

This issue celebrates a 3 years anniversary 🀘

@jstarks
Copy link
Collaborator

@jstarks jstarks commented Nov 16, 2020

Yes. On the other hand, 6 days ago we celebrated the 5 year anniversary of a Windows release with built-in VT support.

So as painful as it is for users stuck on even older versions of Windows, it's unlikely that we'll make any significant improvements to this package. ☹️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants