Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding: add AppendText and AppendBinary #62384

Closed
dsnet opened this issue Aug 30, 2023 · 38 comments
Closed

encoding: add AppendText and AppendBinary #62384

dsnet opened this issue Aug 30, 2023 · 38 comments

Comments

@dsnet
Copy link
Member

dsnet commented Aug 30, 2023

The MarshalText and MarshalBinary methods are a trash factory.
They create a short-lived string that is almost always garbage collectable shortly after creation.
Furthermore, the caller almost always copied the result into another buffer, resulting in further waste.

I propose the addition of the following interfaces to the "encoding" package:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

In addition to the above, we would update all types in the stdlib that implement MarshalText or MarshalBinary to have the Append equivalent (and switch the Marshal implementation to just call the Append variant). Also, we would teach all stdlib libraries that looks for TextMarshaler or BinaryMarshaler to also look for the append (and unmarshal) variant.

\cc @bradfitz @maisem

@dsnet dsnet added the Proposal label Aug 30, 2023
@gopherbot gopherbot added this to the Proposal milestone Aug 30, 2023
@dsnet
Copy link
Member Author

dsnet commented Aug 30, 2023

This is motivated by netip.Addr.MarshalText being a non-trivial amount of slowdown in a large JSON object.

@dsnet
Copy link
Member Author

dsnet commented Aug 30, 2023

Note that I did not propose json.Appender. There are other possible API designs that allow for streaming JSON encoding and decoding which would obviate the need for it. A separate proposal or discussion will be made for this in the future.

@dsnet
Copy link
Member Author

dsnet commented Aug 30, 2023

Analyzing all code on the module proxy from 2023-07-01:

I do not believe there are any backwards compatibility issues with this change.

barista.run@v0.0.0-20230630192130-1694cee7a398/pango/pango.go:95: func (n *Node) AppendText(texts ...string) *Node {
bitbucket.org/afiszone/gotk3@v0.0.0-20200317205801-68a9308a074c/gtk/combo_box.go:263: func (v *ComboBoxText) AppendText(text string) {
code.afis.me/gotk3@v0.0.0-20201225185300-6238108e03d9/gtk/combo_box.go:354: func (v *ComboBoxText) AppendText(text string) {
code.aliyun.com/newrank-tools/document@v0.0.3/api/markdown/markdown.go:47: func (md *Markdown) AppendText(content string, params ...bool) *Markdown {
codeberg.org/japh/psv@v0.2.0/table.go:149: func (tbl *Table) AppendText(textLines ...string) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/extra.go:54: func (t *ExtraTags) AppendBinary(preAllocatedBuf []byte) (buf []byte, err error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/extra.go:124: func (t *ExtraTag) AppendBinary(preAllocatedBuf []byte) ([]byte, error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/output.go:78: func (s *Outputs) AppendBinary(preAllocatedBuf []byte) (data []byte, err error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/p2pool/sidechain/sidedata.go:52: func (b *SideData) AppendBinary(preAllocatedBuf []byte, version ShareVersion) (buf []byte, err error) {
git.milar.in/milarin/tui@v0.0.21/rune.go:46: func (t *Text) AppendText(txt *Text) {
git.tordarus.net/Tordarus/tui@v0.0.15/rune.go:46: func (t *Text) AppendText(txt *Text) {
git.wit.org/wit/gui@v0.8.6/common.go:91: func (n *Node) AppendText(str string) {
gitee.com/scpro/gotk3@v1.0.1/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
gitee.com/zhouhailin/hutool-go@v0.0.2/hutool/io/io_util.go:27: func (f *IoUtil) AppendText(pathname string, text string) error {
github.com/45638654564654/appoptics-apm-go@v0.0.0-20221005200510-e098d2000040/v1/ao/internal/bson/bson.go:43: func (b *Buffer) AppendBinary(k string, v []byte) {
github.com/474420502/libxml2@v0.0.0-20200807040518-4ef6186ae68c/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/BlackMocca/mongo-go-driver@v1.0.4/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/BlackMocca/mongo-go-driver@v1.0.4/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/Existed/libxml2@v0.0.0-20190422014600-3dcec2c44e24/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/FabianWe/etherpadlite-golang@v1.1.0/etherpadlite.go:308: func (pad *EtherpadLite) AppendText(ctx context.Context, padID, text interface{}) (*Response, error) {
github.com/Gipcomp/winapi@v0.0.0-20211028125119-f5f50e8984c7/textedit.go:245: func (te *TextEdit) AppendText(value string) {
github.com/GoTyro/walk@v0.0.0-20220112065045-02bb5eb37497/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/GoTyro/walk@v0.0.0-20220112065045-02bb5eb37497/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/KemalovMaulen/libxml2@v1.0.0/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/Skarm/walk@v0.0.0-20150220080041-98cb48b69104/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/Skarm/walk@v0.0.0-20150220080041-98cb48b69104/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/StephanVerbeeck/walk@v0.0.0-20230202143337-d63a2e470fda/textedit.go:217: func (te *TextEdit) AppendText(value string) {
github.com/StephanVerbeeck/walk@v0.0.0-20230202143337-d63a2e470fda/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/adejoux/nmon2influxdb@v2.1.7-0.20190605081557-84e3405360f7+incompatible/nmon/nmon.go:42: func (nmon *Nmon) AppendText(text string) {
github.com/admpub/spider@v0.0.0-20171028061754-3a892f2f9e33/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/agl/go-gtk@v0.0.0-20151214221645-91c1edb38c24/gtk/gtk.go:5447: func (v *GtkComboBoxText) AppendText(text string) {
github.com/anoshenko/rui@v0.13.0/editView.go:357: func (edit *editViewData) AppendText(text string) {
github.com/appoptics/appoptics-apm-go@v1.16.0/v1/ao/internal/bson/bson.go:43: func (b *Buffer) AppendBinary(k string, v []byte) {
github.com/auroralaboratories/gotk3@v0.0.0-20170323014032-52f9ba079f44/gtk/combo_box.go:229: func (v *ComboBoxText) AppendText(text string) {
github.com/bakharal/mongo-go-driver@v1.10.11/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/bakharal/mongo-go-driver@v1.10.11/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/bennicholls/burl-E@v0.0.0-20190708032244-24ccafc7d477/burl/textbox.go:29: func (t *Textbox) AppendText(txt string) {
github.com/blorticus/tpcli@v0.3.1/tpcli.go:391: func (panel *outputPanel) AppendText(s string) {
github.com/bontibon/go-pipeglade@v0.0.0-20150212114355-0ac764b2f9e2/manipulation.go:141: func (ui *UI) AppendText(widget, text string) {
github.com/bytbox/gogobject@v0.0.0-20120219142239-4e70a8595dcf/gtk-3.0/gtk.go:14947: func (this0 *ComboBoxText) AppendText(text0 string) {
github.com/chai2010/walk@v0.0.2/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/chai2010/walk@v0.0.2/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/clockworkpi/LauncherGoDev@v0.0.0-20230126075719-2743d6b9dc1e/sysgo/UI/textarea.go:80: func (self *Textarea) AppendText(alphabet string) {
github.com/clockworksoul/mediawiki@v0.0.5/edit.go:259: func (w *EditClient) AppendText(s string) *EditClient {
github.com/cloudfly/alarmd@v0.0.0-20191209053936-d369a3264630/pkg/events/channel.go:61: func (spec ChannelSpec) AppendText(data map[string]interface{}) string {
github.com/cloudfly/ecenter@v0.0.0-20191220053526-dfd69a52662f/pkg/events/channel.go:61: func (spec ChannelSpec) AppendText(data map[string]interface{}) string {
github.com/cloudradar-monitoring/frontman@v0.0.0-20210924115943-7c61842a19b8/pkg/winui/winui_logview.go:64: func (lv *LogView) AppendText(value string) {
github.com/coyim/gotk3adapter@v0.0.2/gtk_mock/combo_box_text.go:7: func (*MockComboBoxText) AppendText(v1 string) {
github.com/coyim/gotk3adapter@v0.0.2/gtka/combo_box_text.go:31: func (v *comboBoxText) AppendText(v1 string) {
github.com/coyim/gotk3mocks@v0.0.2/gtk/combo_box_text.go:7: func (m *MockComboBoxText) AppendText(v1 string) {
github.com/d2r2/go-gtk@v0.0.0-20131218031741-e35a6dfb5a85/gtk/gtk.go:5223: func (v *ComboBox) AppendText(text string) {
github.com/d2r2/go-gtk@v0.0.0-20131218031741-e35a6dfb5a85/gtk/gtk.go:5309: func (v *ComboBoxText) AppendText(text string) {
github.com/d2r2/gotk3@v0.0.0-20211023153135-0ceb344f1671/gtk/combobox.go:240: func (v *ComboBoxText) AppendText(text string) {
github.com/davidli2010/gobson_exp@v0.0.0-20161004043538-c651a288efad/bson/bsonarraybuilder.go:103: func (a *BsonArrayBuilder) AppendBinary(value Binary) *BsonArrayBuilder {
github.com/davidli2010/gobson_exp@v0.0.0-20161004043538-c651a288efad/bson/bsonbuilder.go:183: func (b *BsonBuilder) AppendBinary(name string, value Binary) *BsonBuilder {
github.com/diamondburned/gotk4/pkg@v0.0.5/gtk/v3/gtkcomboboxtext.go:235: func (comboBox *ComboBoxText) AppendText(text string) {
github.com/diamondburned/gotk4/pkg@v0.0.5/gtk/v4/gtkcomboboxtext.go:213: func (comboBox *ComboBoxText) AppendText(text string) {
github.com/dimansk12/libxml2@v1.0.0/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:84776: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:245455: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:290453: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:84736: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:245543: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:288912: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:85111: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:246501: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:289885: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/ebitenui/ebitenui@v0.5.4/widget/textarea.go:336: func (l *TextArea) AppendText(value string) {
github.com/electricface/go-gir@v0.0.0-20220119144140-f2fd0857a7da/gtk-3.0/gtk_auto.go:15477: func (v ComboBoxText) AppendText(text string) {
github.com/friendstechday/codeDocToMarkdown@v0.0.0-20150523005008-fc62c7cac2de/converter/java/javaDoc/comment.go:11: func (c *Comment) AppendText(text string) {
github.com/friendstechday/codeDocToMarkdown@v0.0.0-20150523005008-fc62c7cac2de/converter/java/javaDoc/param.go:13: func (p *Param) AppendText(text string) {     
github.com/gangming/walk@v0.0.0-20220825033616-0c0e321abef9/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/gangming/walk@v0.0.0-20220825033616-0c0e321abef9/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx.go:78: func (e *page) AppendBinary(data []byte) (bool, error) {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx.go:94: func (c *Count) AppendBinary([]byte) error {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx_test.go:37: func (e *indexable) AppendBinary(data []byte) (bool, error)  { return false, json.Unmarshal(data, e) }
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx_test.go:52: func (e *indexableSlice) AppendBinary(data []byte) (bool, error) {
github.com/godot-go/godot-go@v0.0.0-20230312082217-9d5a86402ae8/pkg/gdextension/classes.gen.go:341178: func (cx *RichTextLabelImpl) AppendText(
github.com/gofaith/walk@v0.0.0-20200803081117-681a3effcc77/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/gofaith/walk@v0.0.0-20200803081117-681a3effcc77/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/goki/gi@v1.3.19/giv/textbuf.go:707: func (tb *TextBuf) AppendText(text []byte, signal bool) *textbuf.Edit {
github.com/gontikr99/bidbot2@v0.0.0-20190924173835-de946e3c6c1e/controller/gui/logview.go:60: func (lv *LogView) AppendText(value string) {
github.com/gotk3/gotk3@v0.6.2/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/hegeng1212/pholcus@v0.0.0-20230627120450-78db5a490cf2/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/henrylee2cn/pholcus@v1.3.4/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/ianatha/walk@v0.0.0-20200225051714-2c8e61d8d5e8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/ianatha/walk@v0.0.0-20200225051714-2c8e61d8d5e8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/icowan/walk@v0.0.0-20201223064526-8326e90d8f36/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/icowan/walk@v0.0.0-20201223064526-8326e90d8f36/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/ingmardrewing/gomic@v0.0.0-20170621162852-b6f2c68bbe53/fs/html.go:116: func (n *htmlNode) AppendText(txt string) node {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/metal/lists.go:36: func (ar arrayValues) AppendText(v string) error {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/metal/panicValue.go:31: func (p panicValue) AppendText(string) error {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/runtime/internal/gameList.go:126: func (l *gameList) AppendText(v string) {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/runtime/internal/nullList.go:13: func (n nullList) AppendText(string)         {}
github.com/iquanxin/walk@v0.0.0-20230418091138-4b9082def936/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/jannson/walk@v0.0.0-20221010063135-328f21216f11/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/jannson/walk@v0.0.0-20221010063135-328f21216f11/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/jason-wj/pholcus@v0.0.0-20230628053201-2b89c876f2ed/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/joeirimpan/libxml2@v1.0.4/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/johnbartholomew/go-gtk@v0.0.0-20130206210405-e86225ead39e/gtk/gtk.go:4955: func (v *ComboBox) AppendText(text string) {
github.com/johnbartholomew/go-gtk@v0.0.0-20130206210405-e86225ead39e/gtk/gtk.go:5041: func (v *ComboBoxText) AppendText(text string) {
github.com/jopbrown/gtk-sugar@v0.0.0-20191124065551-9a86599f5831/lib/gtk/combo.go:42: func (obj *ComboBoxText) AppendText(text string) {
github.com/jopbrown/gtk-sugar@v0.0.0-20191124065551-9a86599f5831/lib/gtk/combo.gtk2.go:10: func (obj *ComboBox) AppendText(text string) {
github.com/justinyaoqi/pholcus@v0.0.0-20221204035437-fbc1b0707612/gui/logview.go:65: func (lv *LogView) AppendText(value string) {
github.com/jzs/walk@v0.0.0-20181112095039-c7d36e6fcacf/textedit.go:147: func (te *TextEdit) AppendText(value string) {
github.com/jzs/walk@v0.0.0-20181112095039-c7d36e6fcacf/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/karashiiro/kartlobby@v0.0.0-20221019160947-d0596f565030/pkg/colortext/builder.go:11: func (b *ColorTextBuilder) AppendText(text string) *ColorTextBuilder {
github.com/kckrinke/go-gtk@v0.0.0-20180506212049-4cde83b81e6e/gtk/gtk.go:6126: func (v *ComboBox) AppendText(text string) {
github.com/kckrinke/go-gtk@v0.0.0-20180506212049-4cde83b81e6e/gtk/gtk.go:6221: func (v *ComboBoxText) AppendText(text string) {
github.com/ktye/ui@v0.0.0-20210211114936-94158a1502e8/editor/edit.go:47: func (e *Edit) AppendText(s string) {
github.com/kumakichi/walk@v0.0.0-20150122092410-7e83dfcca18a/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/kumakichi/walk@v0.0.0-20150122092410-7e83dfcca18a/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/kvark128/walk@v0.0.0-20230207150301-925603f55833/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/kvark128/walk@v0.0.0-20230207150301-925603f55833/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/lazyshot/gotk3@v0.0.0-20150809002019-08a6d4f8db4c/gtk/gtk.go:2629: func (v *ComboBoxText) AppendText(text string) {
github.com/leaanthony/go-gtk@v0.0.0-20211127021733-72e2f9c2edc1/gtk/gtk.go:6189: func (v *ComboBox) AppendText(text string) {
github.com/leaanthony/go-gtk@v0.0.0-20211127021733-72e2f9c2edc1/gtk/gtk.go:6284: func (v *ComboBoxText) AppendText(text string) {
github.com/lestrrat/go-libxml2@v0.0.0-20180221004755-bb78334e2019/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/lestrrat-go/libxml2@v0.0.0-20210328064523-a6d61dc4e799/dom/node_element.go:64: func (n *Element) AppendText(s string) error {
github.com/liwh011/gonebot@v0.2.1/message.go:44: func (m *Message) AppendText(t string) {
github.com/loyalpartner/go-nigori@v0.0.5/stream.go:31: func (n *nigoriStream) AppendText(str string) {
github.com/lufuhu/gotk3@v0.6.3/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/lxn/walk@v0.0.0-20210112085537-c389da54e794/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/lxn/walk@v0.0.0-20210112085537-c389da54e794/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/mad-day/bsonbox@v0.0.0-20211002061631-2f125da2d253/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/mad-day/bsonbox@v0.0.0-20211002061631-2f125da2d253/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/marcusolsson/passtray@v0.0.0-20150813080859-7d312850bdc9/Godeps/_workspace/src/github.com/mattn/go-gtk/gtk/gtk.go:5310: func (v *ComboBox) AppendText(text string) {
github.com/marcusolsson/passtray@v0.0.0-20150813080859-7d312850bdc9/Godeps/_workspace/src/github.com/mattn/go-gtk/gtk/gtk.go:5396: func (v *ComboBoxText) AppendText(text string) {
github.com/mardukbp/atk@v1.0.3/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/mattn/go-gtk@v0.0.0-20230419095350-e099c1bf7abc/gtk/gtk.go:6188: func (v *ComboBox) AppendText(text string) {
github.com/mattn/go-gtk@v0.0.0-20230419095350-e099c1bf7abc/gtk/gtk.go:6283: func (v *ComboBoxText) AppendText(text string) {
github.com/maurofran/hamcrest4go@v0.2.2/matcher/description.go:70: func (d Description) AppendText(text string) {
github.com/mjm/mpsanity@v0.0.0-20211020031355-e32cecb0bc89/block/builder.go:96: func (b *Builder) AppendText(text string) {
github.com/mrbear258/gotk3@v1.0.0/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/muumlover/walk@v0.0.0-20181212150522-fb9fbbcc6391/textedit.go:151: func (te *TextEdit) AppendText(value string) {
github.com/muumlover/walk@v0.0.0-20181212150522-fb9fbbcc6391/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/natemaia/gotk3@v0.6.4/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/norisatir/go-gtk3@v0.0.0-20120410091437-f6c2ba810f40/gtk3/gtk3.go:11356: func (self *ComboBoxText) AppendText(text string) {
github.com/novln/soba@v0.0.0-20201126173112-c6f1bc371303/encoder_test.go:140: func (encoder *TestEncoder) AppendBinary(value []byte) {}
github.com/novln/soba@v0.0.0-20201126173112-c6f1bc371303/encoder/json/types.go:434: func (encoder *Encoder) AppendBinary(value []byte) {
github.com/oldkingnearby/walk@v1.0.0/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/oldkingnearby/walk@v1.0.0/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/pauldub/go-gtk3@v0.0.0-20120812130113-4604e9480944/gtk3/gtk3.go:11358: func (self *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-2.24.go:16816: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.0.go:18683: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.10.go:19473: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.12.go:19619: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.14.go:19877: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.16.go:19976: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.18.go:20013: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.2.go:18901: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.20.go:20182: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.26.go:20199: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.29.go:20217: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.6.go:20190: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.go:20190: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.4.go:19167: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.6.go:19278: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.8.go:19293: func (recv *ComboBoxText) AppendText(text string) {
github.com/pietroglyph/walk@v0.0.0-20180728074727-d9c93ed8784b/textedit.go:147: func (te *TextEdit) AppendText(value string) {
github.com/pietroglyph/walk@v0.0.0-20180728074727-d9c93ed8784b/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/pirogom/walk@v0.0.0-20230128105705-492944454fc3/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/planetscale/log@v0.0.0-20230320220358-9d4ed51abd8d/encoder.go:195: func (enc *prettyEncoder) AppendBinary(value []byte) {
github.com/pritunl/mongo-go-driver@v1.11.3/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/pritunl/mongo-go-driver@v1.11.3/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/pwiecz/atk@v0.0.0-20210227100443-cb0bf65e43e2/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/pzartem/libxml2@v0.0.0-20200831082948-c3d9892b3a0f/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/qsirwyk/walk@v1.1.1/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/qsirwyk/walk@v1.1.1/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/raceresult/walk@v0.0.4/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/readykit/gd@v0.0.0-20221015225739-25a193a18dd7/api.go:25532: func (gdClass RichTextLabel) AppendText(bbcode string)  { methodCall[struct{}](gdClass.obj.get(), methodRichTextLabel[76], &bbcode) }
github.com/reusee/ggir@v0.0.0-20141013104036-88ce4855c267/gtk/gtk_traits.go:5592: func (self *TraitComboBoxText) AppendText(text string) {
github.com/richardwilkes/cef@v0.5.3/cef/Textfield_gen.go:111: func (d *Textfield) AppendText(text string) {
github.com/romychs/gotk3@v0.0.0-20210328092339-116e09e25f5e/gtk/combobox.go:240: func (v *ComboBoxText) AppendText(text string) {
github.com/rstshardware/walk@v0.0.1/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/rstshardware/walk@v0.0.1/examples/logview/logview.go:53: func (lv *LogView) AppendText(value string) {
github.com/scjtqs2/walk@v0.0.0-20211230061331-3545c43f16e8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/scjtqs2/walk@v0.0.0-20211230061331-3545c43f16e8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_c.go:1059: func (c *ComboBox) AppendText(text string) { ComboBoxAppendText(c, text) }
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_c.go:1109: func (c *ComboBoxText) AppendText(text string) { ComboBoxTextAppendText(c, text) }
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_e.go:194: func (e *Entry) AppendText(text string)            { EntryAppendText(e, text) }
github.com/tailscale/walk@v0.0.0-20230502163740-ea8725bca6d8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/tailscale/walk@v0.0.0-20230502163740-ea8725bca6d8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/tiborvass/uniline@v0.0.0-20150528164013-b39ee5b15151/utils.go:58: func (t text) AppendText(n text) text {
github.com/timelessnesses/l4d2-fmsa-tk@v0.0.0-20230302131554-ce07a2f78b70/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/tmaxmax/go-sse@v0.4.2/message.go:98: func (e *Message) AppendText(chunks ...string) {
github.com/turutcrane/cefingo@v0.5.9/capi/cefingo_gen.go:27863: func (self *CTextfieldT) AppendText(
github.com/vikulin/walk@v0.0.0-20210329133517-4f03df2e225a/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/vikulin/walk@v0.0.0-20210329133517-4f03df2e225a/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/visualfc/atk@v1.2.3/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/wangch/walk@v0.0.0-20150802023425-cdcaffe9ed2d/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/wangch/walk@v0.0.0-20150802023425-cdcaffe9ed2d/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/wayf-dk/go-libxml2@v0.0.0-20210718205101-95691c57c1ac/dom/node_element.go:65: func (n *Element) AppendText(s string) error {
github.com/weili71/go-filex@v0.0.0-20230321132739-7a825b3f5691/filex.go:200: func (f *Filex) AppendText(file *os.File, text string) (int, error) {
github.com/whtiehack/wingui@v0.0.6/edit.go:50: func (e *Edit) AppendText(value string) {
github.com/winxxp/walk@v0.0.0-20200901104446-3e7ddcbafc7d/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/winxxp/walk@v0.0.0-20200901104446-3e7ddcbafc7d/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/xlplbo/walk@v0.0.0-20201127063646-ea1cc4883b4d/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/xlplbo/walk@v0.0.0-20201127063646-ea1cc4883b4d/examples/logview/logview.go:38: func (lv *LogView) AppendText(value string) {
github.com/yamnikov-oleg/go-gtk@v0.0.0-20160312170510-bc91c0fbef0c/gtk/gtk.go:5343: func (v *ComboBox) AppendText(text string) {
github.com/yamnikov-oleg/go-gtk@v0.0.0-20160312170510-bc91c0fbef0c/gtk/gtk.go:5430: func (v *ComboBoxText) AppendText(text string) {
github.com/zhangweiii/pholcus@v0.0.0-20200228101341-71bf9bad259b/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/zhangyiming748/sendEmailAlert@v0.0.14/alert.go:67: func (i *Info) AppendText(s string) {
github.com/zurek87/go-gtk3@v0.0.0-20171203223943-bec742635bb7/gtk3/gtk3.go:5867: func (v *ComboBoxText) AppendText(text string) {
github.com/zxjsdp/specimen-go@v1.9.4/gui/windows/logview.go:63: func (lv *LogView) AppendText(value string) {
gitlab.com/abibino-lab/mobal@v0.0.0-20170907170701-2d91a9572e70/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
gitlab.com/gitlab-org/security-products/analyzers/common/table@v0.0.0-20181219150649-131c639c2176/table.go:30: func (t *Table) AppendText(text string) {
gitlab.com/gitlab-org/security-products/analyzers/common/table/v2@v2.0.0-20201123204353-3743d5d8b4cc/table.go:30: func (t *Table) AppendText(text string) {
go.mongodb.org/mongo-driver@v1.12.0/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
go.mongodb.org/mongo-driver@v1.12.0/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
modernc.org/atk@v1.4.3/text.go:410: func (w *Text) AppendText(text string) error {

@mateusz834
Copy link
Member

This is motivated by netip.Addr.MarshalText being a non-trivial amount of slowdown in a large JSON object.

Just so you now there is netip.Addr.AppendTo, that does the same thing as MarshalText, but appends.

@dsnet
Copy link
Member Author

dsnet commented Aug 31, 2023

There are unfortunately two problems with AppendTo:

  • It doesn't return an error, and so can't implement equivalent semantics as MarshalText
  • It's naming is ambiguous with regard to whether it does "binary" or "textual" marshaling

@mateusz834
Copy link
Member

@dsnet Yes I agree, just wanted to point that there will be now two methods that do the same thing on netip.Addr, probably after this proposal gets accepted the AppendTo should be deprecated.

@mvdan
Copy link
Member

mvdan commented Aug 31, 2023

Scratching my head and wondering how this hasn't been proposed before :) sounds great to me.

@flimzy
Copy link
Contributor

flimzy commented Aug 31, 2023

What are the possible security implications wrt:

// Implementations must not retain b, nor mutate any bytes within b[:len(b)].

I can imagine a malicious, or sloppy, library not following this rule, and introducing nastiness.

Given the other vectors for "nastiness" that already exist throughout the stdlib when using untrusted code, probably not a meaningful consideration, but it does seem a bit icky to me, to expose some other code's data to arbitrary code this way.

@dsnet
Copy link
Member Author

dsnet commented Aug 31, 2023

Given the other vectors for "nastiness" that already exist throughout the stdlib when using untrusted code, probably not a meaningful consideration, but it does seem a bit icky to me, to expose some other code's data to arbitrary code this way.

Agreed. That's the strength of the MarshalText API. It's much harder to hold wrong.

That said, two of the top-three interfaces in Go by far are io.Reader and io.Writer, both of which expose a buffer to the implementation and document:

Implementations must not retain p.

I don't think we're making the situation any worse since encoding.AppendText will be multiple orders of magnitude less popular than io.Reader or io.Writer.

@seebs
Copy link
Contributor

seebs commented Sep 1, 2023

This seems very desireable, the large allocations can be a problem. Although I think in a lot of the cases that are particularly messy, append might not help, because i really need to do an encoder to an io.Writer? But I would still very much like to have this, because iterative loops that are marshaling a bunch of things but only need one at a time are pretty common.

@mateusz834
Copy link
Member

@seebs That is true that something like this would be a nice alternative.

type TextWriter interface {
    WriteText(b io.Writer) error
}

But, this as as side effect would cause another bunch of memory allocations inside each implementation of WriteText, because escape analysis is not great for this use case.

@dsnet
Copy link
Member Author

dsnet commented Sep 1, 2023

Any encoder writing eventually to an io.Writer almost certainly have an intermediate buffer on hand that it can pass to AppendText and then pass eventually to io.Writer.Write.

@yerden
Copy link

yerden commented Sep 22, 2023

Upvoting. I cannot imagine any heavily loaded application tinkering with bytes and text that wouldn't implement an interface of similar semantics.

@mitar
Copy link
Contributor

mitar commented Oct 6, 2023

nor mutate any bytes within b[:len(b)].

Why, performance wise, would that not be allowed? Maybe we can open additional use cases if we do allow that?

@dsnet
Copy link
Member Author

dsnet commented Oct 6, 2023

Why, performance wise, would that not be allowed?

For a function named "Append", it just seems highly suspicious for it to mutate data within b[:len(b)]. The verb "append" implies only modifying data within b[len(b):cap(b)].

A performance argument could be made in the reverse. The caller of AppendText could in theory spin off a goroutine to do some form of read-only validation within b[:len(b)]. Mutating that section would result in a race condition.

@mitar
Copy link
Contributor

mitar commented Oct 6, 2023

For a function named "Append", it just seems highly suspicious for it to mutate data within b[:len(b)]. The verb "append" implies only modifying data within b[len(b):cap(b)].

So should we rename it then? Like MutateText(b []byte) ([]byte, error)?

A performance argument could be made in the reverse. The caller of AppendText could in theory spin off a goroutine to do some form of read-only validation within b[:len(b)]. Mutating that section would result in a race condition.

I think this is handled by the "Implementations must not retain b", if you make a goroutine, you are retaining b.

@dsnet
Copy link
Member Author

dsnet commented Oct 6, 2023

So should we rename it then?

I'm arguing that we should keep it named AppendText and also keep the documentation that implementations shouldn't mutate b[:len(b)].

I think this is handled by the "Implementations must not retain b", if you make a goroutine, you are retaining b.

In my example, the asynchronous validation logic is done by the caller of AppendText, not by the implementation of AppendText.

@mitar
Copy link
Contributor

mitar commented Oct 7, 2023

@dsnet Thanks for the explanation!

@mvdan
Copy link
Member

mvdan commented Apr 2, 2024

@ianlancetaylor could I nudge this proposal forward? It would help with the encoding/json/v2 efforts, and it seems to have reasonable consensus :)

@rsc
Copy link
Contributor

rsc commented Apr 24, 2024

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented May 8, 2024

Have all remaining concerns about this proposal been addressed?

The proposal is to add to package encoding:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

And then these types would get new methods alongside their existing MarshalText and MarshalBinary methods:

go1.18.txt:pkg net/netip, method (Addr) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalText() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalText() ([]uint8, error)
go1.2.txt:pkg encoding, type TextMarshaler interface { MarshalText }
go1.2.txt:pkg encoding, type TextMarshaler interface, MarshalText() ([]uint8, error)
go1.2.txt:pkg net, method (IP) MarshalText() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalText() ([]uint8, error)
go1.21.txt:pkg log/slog, method (Level) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg log/slog, method (*LevelVar) MarshalText() ([]uint8, error) #56345
go1.21.txt:pkg regexp, method (*Regexp) MarshalText() ([]uint8, error) #46159
go1.3.txt:pkg math/big, method (*Int) MarshalText() ([]uint8, error)
go1.3.txt:pkg math/big, method (*Rat) MarshalText() ([]uint8, error)
go1.6.txt:pkg math/big, method (*Float) MarshalText() ([]uint8, error)

go1.18.txt:pkg net/netip, method (Addr) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (AddrPort) MarshalBinary() ([]uint8, error)
go1.18.txt:pkg net/netip, method (Prefix) MarshalBinary() ([]uint8, error)
go1.2.txt:pkg encoding, type BinaryMarshaler interface { MarshalBinary }
go1.2.txt:pkg encoding, type BinaryMarshaler interface, MarshalBinary() ([]uint8, error)
go1.2.txt:pkg time, method (Time) MarshalBinary() ([]uint8, error)
go1.22.txt:pkg math/rand/v2, method (*ChaCha8) MarshalBinary() ([]uint8, error) #61716
go1.22.txt:pkg math/rand/v2, method (*PCG) MarshalBinary() ([]uint8, error) #61716
go1.8.txt:pkg net/url, method (*URL) MarshalBinary() ([]uint8, error)

Note that as long as a type has MarshalText, adding AppendText later is not a breaking change, since code should fall back to MarshalText anyway. Same for Binary. So adding these methods is not strictly speaking necessary to happen in Go 1.23 even if the new interfaces are defined in Go 1.23.

@mateusz834
Copy link
Member

mateusz834 commented May 9, 2024

And then these types would get new methods alongside their existing MarshalText and MarshalBinary methods

Probably also the crypto hashes would need AppendBinary, but only accessable through interface assertion.

EDIT: Also we need append methods for #66249.

@rsc
Copy link
Contributor

rsc commented May 15, 2024

It seems like these are a good idea, but it's a lot of new API to get right. Since the Go 1.23 freeze is next week, let's hold off on any of these until the Go 1.24 cycle.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/605758 mentions this issue: math/big,regexp: implement the encoding.TextAppender interface

gopherbot pushed a commit that referenced this issue Aug 15, 2024
For #62384

Change-Id: I1557704c6a0f9c6f3b9aad001374dd5cdbc99065
GitHub-Last-Rev: c258d18
GitHub-Pull-Request: #68893
Reviewed-on: https://go-review.googlesource.com/c/go/+/605758
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
"Time.MarshalText" could also gain some performance improvements:

goos: linux
goarch: amd64
pkg: time
cpu: Intel(R) Core(TM) i5-10200H CPU @ 2.40GHz
              │     old      │                 new                 │
              │    sec/op    │   sec/op     vs base                │
MarshalText-8   103.20n ± 1%   67.41n ± 1%  -34.68% (p=0.000 n=10)

              │    old     │                new                 │
              │    B/op    │   B/op     vs base                 │
MarshalText-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

              │    old     │                 new                 │
              │ allocs/op  │ allocs/op   vs base                 │
MarshalText-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/606655 mentions this issue: time: implement the encoding.(Binary|Text)Appender for Time

apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
gopherbot pushed a commit that referenced this issue Aug 20, 2024
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For #62384

Change-Id: I320421878a341abf8d668fd57b27292cdfa61330
GitHub-Last-Rev: e04f8df
GitHub-Pull-Request: #68942
Reviewed-on: https://go-review.googlesource.com/c/go/+/606655
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
apocelipes added a commit to apocelipes/go that referenced this issue Aug 21, 2024
Implement the encoding.(Binary|Text)Appender interfaces for "x509.OID".

Implement the encoding.BinaryAppender interface for "rand/v2.PCG" and
"rand/v2.ChaCha8".

"rand/v2.ChaCha8.MarshalBinary" alse gains some performance benefits:

                           │     old      │                 new                 │
                           │    sec/op    │   sec/op     vs base                │
ChaCha8MarshalBinary-8       33.730n ± 2%   9.786n ± 1%  -70.99% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8    99.86n ± 1%   17.79n ± 0%  -82.18% (p=0.000 n=10)
geomean                       58.04n        13.19n       -77.27%

                           │    old     │                  new                   │
                           │    B/op    │   B/op     vs base                     │
ChaCha8MarshalBinary-8       48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   83.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                           │    old     │                   new                   │
                           │ allocs/op  │ allocs/op   vs base                     │
ChaCha8MarshalBinary-8       1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/607079 mentions this issue: crypto/x509,math/rand/v2: implement the encoding.(Binary|Text)Appender

gopherbot pushed a commit that referenced this issue Aug 21, 2024
Implement the encoding.(Binary|Text)Appender interfaces for "x509.OID".

Implement the encoding.BinaryAppender interface for "rand/v2.PCG" and "rand/v2.ChaCha8".

"rand/v2.ChaCha8.MarshalBinary" alse gains some performance benefits:

                           │     old      │                 new                 │
                           │    sec/op    │   sec/op     vs base                │
ChaCha8MarshalBinary-8       33.730n ± 2%   9.786n ± 1%  -70.99% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8    99.86n ± 1%   17.79n ± 0%  -82.18% (p=0.000 n=10)
geomean                       58.04n        13.19n       -77.27%

                           │    old     │                  new                   │
                           │    B/op    │   B/op     vs base                     │
ChaCha8MarshalBinary-8       48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   83.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                           │    old     │                   new                   │
                           │ allocs/op  │ allocs/op   vs base                     │
ChaCha8MarshalBinary-8       1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For #62384

Change-Id: I604bde6dad90a916012909c7260f4bb06dcf5c0a
GitHub-Last-Rev: 78abf9c
GitHub-Pull-Request: #68987
Reviewed-on: https://go-review.googlesource.com/c/go/+/607079
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
Implement the encoding.TextAppender for "net.IP".

Implament the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/607520 mentions this issue: net,net/netip: implement the encoding.(Binary|Text)Appender

apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
Implement the encoding.TextAppender for "net.IP".

Implament the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
Implement the encoding.TextAppender interface for "net.IP".

Implament the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Sep 3, 2024
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Sep 19, 2024
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Sep 20, 2024
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                          │     old      │                 new                 │
                          │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8         66.06n ± 1%   14.55n ± 1%  -77.97% (p=0.000 n=10)
IPMarshalText/IPv6-8        117.00n ± 1%   63.18n ± 1%  -46.00% (p=0.000 n=10)
IPMarshalText/IPv6_long-8    137.8n ± 1%   111.3n ± 1%  -19.27% (p=0.000 n=10)
geomean                      102.1n        46.77n       -54.21%

                          │    old     │                   new                   │
                          │    B/op    │    B/op     vs base                     │
IPMarshalText/IPv4-8        32.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8        48.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6_long-8   96.00 ± 0%   48.00 ± 0%   -50.00% (p=0.000 n=10)

                          │    old     │                   new                   │
                          │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8        2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8        2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6_long-8   2.000 ± 0%   1.000 ± 0%   -50.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Accepted
Development

Successfully merging a pull request may close this issue.

9 participants