Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding: add AppendText and AppendBinary #62384

Closed
dsnet opened this issue Aug 30, 2023 · 38 comments
Closed

encoding: add AppendText and AppendBinary #62384

dsnet opened this issue Aug 30, 2023 · 38 comments
Labels
Performance Proposal Proposal-Accepted
Milestone
Backlog

Comments

@dsnet
Copy link
Member

dsnet commented Aug 30, 2023 •
edited
Loading

The MarshalText and MarshalBinary methods are a trash factory.
They create a short-lived string that is almost always garbage collectable shortly after creation.
Furthermore, the caller almost always copied the result into another buffer, resulting in further waste.

I propose the addition of the following interfaces to the "encoding" package:

// TextAppender is the interface implemented by an object
// that can append the textual representation of itself.
// If a type implements both [TextAppender] and [TextMarshaler],
// then v.MarshalText() must be semantically identical to v.AppendText(nil).
type TextAppender interface {
    // AppendText appends the textual representation of itself to the end of b
    // (allocating a larger slice if necessary) and returns the updated slice.
    //
    // Implementations must not retain b, nor mutate any bytes within b[:len(b)].
    AppendText(b []byte) ([]byte, error)
}

type BinaryAppender interface {
    AppendBinary([]byte) ([]byte, error)
}

In addition to the above, we would update all types in the stdlib that implement MarshalText or MarshalBinary to have the Append equivalent (and switch the Marshal implementation to just call the Append variant). Also, we would teach all stdlib libraries that looks for TextMarshaler or BinaryMarshaler to also look for the append (and unmarshal) variant.

\cc @bradfitz @maisem
@dsnet dsnet added the Proposal label Aug 30, 2023
@gopherbot gopherbot added this to the Proposal milestone Aug 30, 2023
@dsnet
Copy link
Member Author

dsnet commented Aug 30, 2023

This is motivated by netip.Addr.MarshalText being a non-trivial amount of slowdown in a large JSON object.

@dsnet dsnet mentioned this issue Aug 30, 2023
Closed
@dsnet
Copy link
Member Author

dsnet commented Aug 30, 2023

Note that I did not propose json.Appender. There are other possible API designs that allow for streaming JSON encoding and decoding which would obviate the need for it. A separate proposal or discussion will be made for this in the future.

@ianlancetaylor ianlancetaylor moved this to Incoming in Proposals Aug 30, 2023
@dsnet
Copy link
Member Author

dsnet commented Aug 30, 2023

Analyzing all code on the module proxy from 2023-07-01:

I do not believe there are any backwards compatibility issues with this change.

barista.run@v0.0.0-20230630192130-1694cee7a398/pango/pango.go:95: func (n *Node) AppendText(texts ...string) *Node {
bitbucket.org/afiszone/gotk3@v0.0.0-20200317205801-68a9308a074c/gtk/combo_box.go:263: func (v *ComboBoxText) AppendText(text string) {
code.afis.me/gotk3@v0.0.0-20201225185300-6238108e03d9/gtk/combo_box.go:354: func (v *ComboBoxText) AppendText(text string) {
code.aliyun.com/newrank-tools/document@v0.0.3/api/markdown/markdown.go:47: func (md *Markdown) AppendText(content string, params ...bool) *Markdown {
codeberg.org/japh/psv@v0.2.0/table.go:149: func (tbl *Table) AppendText(textLines ...string) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/extra.go:54: func (t *ExtraTags) AppendBinary(preAllocatedBuf []byte) (buf []byte, err error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/extra.go:124: func (t *ExtraTag) AppendBinary(preAllocatedBuf []byte) ([]byte, error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/monero/transaction/output.go:78: func (s *Outputs) AppendBinary(preAllocatedBuf []byte) (data []byte, err error) {
git.gammaspectra.live/P2Pool/p2pool-observer@v0.0.0-20230605214634-bad1e302c8c8/p2pool/sidechain/sidedata.go:52: func (b *SideData) AppendBinary(preAllocatedBuf []byte, version ShareVersion) (buf []byte, err error) {
git.milar.in/milarin/tui@v0.0.21/rune.go:46: func (t *Text) AppendText(txt *Text) {
git.tordarus.net/Tordarus/tui@v0.0.15/rune.go:46: func (t *Text) AppendText(txt *Text) {
git.wit.org/wit/gui@v0.8.6/common.go:91: func (n *Node) AppendText(str string) {
gitee.com/scpro/gotk3@v1.0.1/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
gitee.com/zhouhailin/hutool-go@v0.0.2/hutool/io/io_util.go:27: func (f *IoUtil) AppendText(pathname string, text string) error {
github.com/45638654564654/appoptics-apm-go@v0.0.0-20221005200510-e098d2000040/v1/ao/internal/bson/bson.go:43: func (b *Buffer) AppendBinary(k string, v []byte) {
github.com/474420502/libxml2@v0.0.0-20200807040518-4ef6186ae68c/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/BlackMocca/mongo-go-driver@v1.0.4/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/BlackMocca/mongo-go-driver@v1.0.4/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/Existed/libxml2@v0.0.0-20190422014600-3dcec2c44e24/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/FabianWe/etherpadlite-golang@v1.1.0/etherpadlite.go:308: func (pad *EtherpadLite) AppendText(ctx context.Context, padID, text interface{}) (*Response, error) {
github.com/Gipcomp/winapi@v0.0.0-20211028125119-f5f50e8984c7/textedit.go:245: func (te *TextEdit) AppendText(value string) {
github.com/GoTyro/walk@v0.0.0-20220112065045-02bb5eb37497/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/GoTyro/walk@v0.0.0-20220112065045-02bb5eb37497/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/KemalovMaulen/libxml2@v1.0.0/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/Skarm/walk@v0.0.0-20150220080041-98cb48b69104/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/Skarm/walk@v0.0.0-20150220080041-98cb48b69104/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/StephanVerbeeck/walk@v0.0.0-20230202143337-d63a2e470fda/textedit.go:217: func (te *TextEdit) AppendText(value string) {
github.com/StephanVerbeeck/walk@v0.0.0-20230202143337-d63a2e470fda/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/adejoux/nmon2influxdb@v2.1.7-0.20190605081557-84e3405360f7+incompatible/nmon/nmon.go:42: func (nmon *Nmon) AppendText(text string) {
github.com/admpub/spider@v0.0.0-20171028061754-3a892f2f9e33/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/agl/go-gtk@v0.0.0-20151214221645-91c1edb38c24/gtk/gtk.go:5447: func (v *GtkComboBoxText) AppendText(text string) {
github.com/anoshenko/rui@v0.13.0/editView.go:357: func (edit *editViewData) AppendText(text string) {
github.com/appoptics/appoptics-apm-go@v1.16.0/v1/ao/internal/bson/bson.go:43: func (b *Buffer) AppendBinary(k string, v []byte) {
github.com/auroralaboratories/gotk3@v0.0.0-20170323014032-52f9ba079f44/gtk/combo_box.go:229: func (v *ComboBoxText) AppendText(text string) {
github.com/bakharal/mongo-go-driver@v1.10.11/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/bakharal/mongo-go-driver@v1.10.11/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/bennicholls/burl-E@v0.0.0-20190708032244-24ccafc7d477/burl/textbox.go:29: func (t *Textbox) AppendText(txt string) {
github.com/blorticus/tpcli@v0.3.1/tpcli.go:391: func (panel *outputPanel) AppendText(s string) {
github.com/bontibon/go-pipeglade@v0.0.0-20150212114355-0ac764b2f9e2/manipulation.go:141: func (ui *UI) AppendText(widget, text string) {
github.com/bytbox/gogobject@v0.0.0-20120219142239-4e70a8595dcf/gtk-3.0/gtk.go:14947: func (this0 *ComboBoxText) AppendText(text0 string) {
github.com/chai2010/walk@v0.0.2/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/chai2010/walk@v0.0.2/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/clockworkpi/LauncherGoDev@v0.0.0-20230126075719-2743d6b9dc1e/sysgo/UI/textarea.go:80: func (self *Textarea) AppendText(alphabet string) {
github.com/clockworksoul/mediawiki@v0.0.5/edit.go:259: func (w *EditClient) AppendText(s string) *EditClient {
github.com/cloudfly/alarmd@v0.0.0-20191209053936-d369a3264630/pkg/events/channel.go:61: func (spec ChannelSpec) AppendText(data map[string]interface{}) string {
github.com/cloudfly/ecenter@v0.0.0-20191220053526-dfd69a52662f/pkg/events/channel.go:61: func (spec ChannelSpec) AppendText(data map[string]interface{}) string {
github.com/cloudradar-monitoring/frontman@v0.0.0-20210924115943-7c61842a19b8/pkg/winui/winui_logview.go:64: func (lv *LogView) AppendText(value string) {
github.com/coyim/gotk3adapter@v0.0.2/gtk_mock/combo_box_text.go:7: func (*MockComboBoxText) AppendText(v1 string) {
github.com/coyim/gotk3adapter@v0.0.2/gtka/combo_box_text.go:31: func (v *comboBoxText) AppendText(v1 string) {
github.com/coyim/gotk3mocks@v0.0.2/gtk/combo_box_text.go:7: func (m *MockComboBoxText) AppendText(v1 string) {
github.com/d2r2/go-gtk@v0.0.0-20131218031741-e35a6dfb5a85/gtk/gtk.go:5223: func (v *ComboBox) AppendText(text string) {
github.com/d2r2/go-gtk@v0.0.0-20131218031741-e35a6dfb5a85/gtk/gtk.go:5309: func (v *ComboBoxText) AppendText(text string) {
github.com/d2r2/gotk3@v0.0.0-20211023153135-0ceb344f1671/gtk/combobox.go:240: func (v *ComboBoxText) AppendText(text string) {
github.com/davidli2010/gobson_exp@v0.0.0-20161004043538-c651a288efad/bson/bsonarraybuilder.go:103: func (a *BsonArrayBuilder) AppendBinary(value Binary) *BsonArrayBuilder {
github.com/davidli2010/gobson_exp@v0.0.0-20161004043538-c651a288efad/bson/bsonbuilder.go:183: func (b *BsonBuilder) AppendBinary(name string, value Binary) *BsonBuilder {
github.com/diamondburned/gotk4/pkg@v0.0.5/gtk/v3/gtkcomboboxtext.go:235: func (comboBox *ComboBoxText) AppendText(text string) {
github.com/diamondburned/gotk4/pkg@v0.0.5/gtk/v4/gtkcomboboxtext.go:213: func (comboBox *ComboBoxText) AppendText(text string) {
github.com/dimansk12/libxml2@v1.0.0/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:84776: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:245455: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_darwin.go:290453: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:84736: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:245543: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_linux.go:288912: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:85111: func (arg1 SwigClassTextEntry) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:246501: func (arg1 SwigClassStyledTextCtrl) AppendText(arg2 string) {
github.com/dontpanic92/wxGo@v0.0.0-20181107055819-fbf544604121/wx/wx_windows.go:289885: func (arg1 SwigClassRichTextCtrl) AppendText(arg2 string) {
github.com/ebitenui/ebitenui@v0.5.4/widget/textarea.go:336: func (l *TextArea) AppendText(value string) {
github.com/electricface/go-gir@v0.0.0-20220119144140-f2fd0857a7da/gtk-3.0/gtk_auto.go:15477: func (v ComboBoxText) AppendText(text string) {
github.com/friendstechday/codeDocToMarkdown@v0.0.0-20150523005008-fc62c7cac2de/converter/java/javaDoc/comment.go:11: func (c *Comment) AppendText(text string) {
github.com/friendstechday/codeDocToMarkdown@v0.0.0-20150523005008-fc62c7cac2de/converter/java/javaDoc/param.go:13: func (p *Param) AppendText(text string) {     
github.com/gangming/walk@v0.0.0-20220825033616-0c0e321abef9/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/gangming/walk@v0.0.0-20220825033616-0c0e321abef9/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx.go:78: func (e *page) AppendBinary(data []byte) (bool, error) {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx.go:94: func (c *Count) AppendBinary([]byte) error {
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx_test.go:37: func (e *indexable) AppendBinary(data []byte) (bool, error)  { return false, json.Unmarshal(data, e) }
github.com/gimalay/binx@v0.0.0-20200623152039-84df950e8e35/binx_test.go:52: func (e *indexableSlice) AppendBinary(data []byte) (bool, error) {
github.com/godot-go/godot-go@v0.0.0-20230312082217-9d5a86402ae8/pkg/gdextension/classes.gen.go:341178: func (cx *RichTextLabelImpl) AppendText(
github.com/gofaith/walk@v0.0.0-20200803081117-681a3effcc77/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/gofaith/walk@v0.0.0-20200803081117-681a3effcc77/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/goki/gi@v1.3.19/giv/textbuf.go:707: func (tb *TextBuf) AppendText(text []byte, signal bool) *textbuf.Edit {
github.com/gontikr99/bidbot2@v0.0.0-20190924173835-de946e3c6c1e/controller/gui/logview.go:60: func (lv *LogView) AppendText(value string) {
github.com/gotk3/gotk3@v0.6.2/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/hegeng1212/pholcus@v0.0.0-20230627120450-78db5a490cf2/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/henrylee2cn/pholcus@v1.3.4/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/ianatha/walk@v0.0.0-20200225051714-2c8e61d8d5e8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/ianatha/walk@v0.0.0-20200225051714-2c8e61d8d5e8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/icowan/walk@v0.0.0-20201223064526-8326e90d8f36/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/icowan/walk@v0.0.0-20201223064526-8326e90d8f36/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/ingmardrewing/gomic@v0.0.0-20170621162852-b6f2c68bbe53/fs/html.go:116: func (n *htmlNode) AppendText(txt string) node {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/metal/lists.go:36: func (ar arrayValues) AppendText(v string) error {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/metal/panicValue.go:31: func (p panicValue) AppendText(string) error {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/runtime/internal/gameList.go:126: func (l *gameList) AppendText(v string) {
github.com/ionous/sashimi@v0.0.0-20160915000703-9034458fa291/runtime/internal/nullList.go:13: func (n nullList) AppendText(string)         {}
github.com/iquanxin/walk@v0.0.0-20230418091138-4b9082def936/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/jannson/walk@v0.0.0-20221010063135-328f21216f11/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/jannson/walk@v0.0.0-20221010063135-328f21216f11/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/jason-wj/pholcus@v0.0.0-20230628053201-2b89c876f2ed/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/joeirimpan/libxml2@v1.0.4/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/johnbartholomew/go-gtk@v0.0.0-20130206210405-e86225ead39e/gtk/gtk.go:4955: func (v *ComboBox) AppendText(text string) {
github.com/johnbartholomew/go-gtk@v0.0.0-20130206210405-e86225ead39e/gtk/gtk.go:5041: func (v *ComboBoxText) AppendText(text string) {
github.com/jopbrown/gtk-sugar@v0.0.0-20191124065551-9a86599f5831/lib/gtk/combo.go:42: func (obj *ComboBoxText) AppendText(text string) {
github.com/jopbrown/gtk-sugar@v0.0.0-20191124065551-9a86599f5831/lib/gtk/combo.gtk2.go:10: func (obj *ComboBox) AppendText(text string) {
github.com/justinyaoqi/pholcus@v0.0.0-20221204035437-fbc1b0707612/gui/logview.go:65: func (lv *LogView) AppendText(value string) {
github.com/jzs/walk@v0.0.0-20181112095039-c7d36e6fcacf/textedit.go:147: func (te *TextEdit) AppendText(value string) {
github.com/jzs/walk@v0.0.0-20181112095039-c7d36e6fcacf/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/karashiiro/kartlobby@v0.0.0-20221019160947-d0596f565030/pkg/colortext/builder.go:11: func (b *ColorTextBuilder) AppendText(text string) *ColorTextBuilder {
github.com/kckrinke/go-gtk@v0.0.0-20180506212049-4cde83b81e6e/gtk/gtk.go:6126: func (v *ComboBox) AppendText(text string) {
github.com/kckrinke/go-gtk@v0.0.0-20180506212049-4cde83b81e6e/gtk/gtk.go:6221: func (v *ComboBoxText) AppendText(text string) {
github.com/ktye/ui@v0.0.0-20210211114936-94158a1502e8/editor/edit.go:47: func (e *Edit) AppendText(s string) {
github.com/kumakichi/walk@v0.0.0-20150122092410-7e83dfcca18a/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/kumakichi/walk@v0.0.0-20150122092410-7e83dfcca18a/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/kvark128/walk@v0.0.0-20230207150301-925603f55833/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/kvark128/walk@v0.0.0-20230207150301-925603f55833/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/lazyshot/gotk3@v0.0.0-20150809002019-08a6d4f8db4c/gtk/gtk.go:2629: func (v *ComboBoxText) AppendText(text string) {
github.com/leaanthony/go-gtk@v0.0.0-20211127021733-72e2f9c2edc1/gtk/gtk.go:6189: func (v *ComboBox) AppendText(text string) {
github.com/leaanthony/go-gtk@v0.0.0-20211127021733-72e2f9c2edc1/gtk/gtk.go:6284: func (v *ComboBoxText) AppendText(text string) {
github.com/lestrrat/go-libxml2@v0.0.0-20180221004755-bb78334e2019/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/lestrrat-go/libxml2@v0.0.0-20210328064523-a6d61dc4e799/dom/node_element.go:64: func (n *Element) AppendText(s string) error {
github.com/liwh011/gonebot@v0.2.1/message.go:44: func (m *Message) AppendText(t string) {
github.com/loyalpartner/go-nigori@v0.0.5/stream.go:31: func (n *nigoriStream) AppendText(str string) {
github.com/lufuhu/gotk3@v0.6.3/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/lxn/walk@v0.0.0-20210112085537-c389da54e794/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/lxn/walk@v0.0.0-20210112085537-c389da54e794/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/mad-day/bsonbox@v0.0.0-20211002061631-2f125da2d253/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/mad-day/bsonbox@v0.0.0-20211002061631-2f125da2d253/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/marcusolsson/passtray@v0.0.0-20150813080859-7d312850bdc9/Godeps/_workspace/src/github.com/mattn/go-gtk/gtk/gtk.go:5310: func (v *ComboBox) AppendText(text string) {
github.com/marcusolsson/passtray@v0.0.0-20150813080859-7d312850bdc9/Godeps/_workspace/src/github.com/mattn/go-gtk/gtk/gtk.go:5396: func (v *ComboBoxText) AppendText(text string) {
github.com/mardukbp/atk@v1.0.3/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/mattn/go-gtk@v0.0.0-20230419095350-e099c1bf7abc/gtk/gtk.go:6188: func (v *ComboBox) AppendText(text string) {
github.com/mattn/go-gtk@v0.0.0-20230419095350-e099c1bf7abc/gtk/gtk.go:6283: func (v *ComboBoxText) AppendText(text string) {
github.com/maurofran/hamcrest4go@v0.2.2/matcher/description.go:70: func (d Description) AppendText(text string) {
github.com/mjm/mpsanity@v0.0.0-20211020031355-e32cecb0bc89/block/builder.go:96: func (b *Builder) AppendText(text string) {
github.com/mrbear258/gotk3@v1.0.0/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/muumlover/walk@v0.0.0-20181212150522-fb9fbbcc6391/textedit.go:151: func (te *TextEdit) AppendText(value string) {
github.com/muumlover/walk@v0.0.0-20181212150522-fb9fbbcc6391/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/natemaia/gotk3@v0.6.4/gtk/combo_box.go:364: func (v *ComboBoxText) AppendText(text string) {
github.com/norisatir/go-gtk3@v0.0.0-20120410091437-f6c2ba810f40/gtk3/gtk3.go:11356: func (self *ComboBoxText) AppendText(text string) {
github.com/novln/soba@v0.0.0-20201126173112-c6f1bc371303/encoder_test.go:140: func (encoder *TestEncoder) AppendBinary(value []byte) {}
github.com/novln/soba@v0.0.0-20201126173112-c6f1bc371303/encoder/json/types.go:434: func (encoder *Encoder) AppendBinary(value []byte) {
github.com/oldkingnearby/walk@v1.0.0/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/oldkingnearby/walk@v1.0.0/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/pauldub/go-gtk3@v0.0.0-20120812130113-4604e9480944/gtk3/gtk3.go:11358: func (self *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-2.24.go:16816: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.0.go:18683: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.10.go:19473: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.12.go:19619: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.14.go:19877: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.16.go:19976: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.18.go:20013: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.2.go:18901: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.20.go:20182: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.26.go:20199: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.29.go:20217: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.6.go:20190: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.22.go:20190: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.4.go:19167: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.6.go:19278: func (recv *ComboBoxText) AppendText(text string) {
github.com/pekim/gobbi@v0.1.1/lib/gtk/v-3.8.go:19293: func (recv *ComboBoxText) AppendText(text string) {
github.com/pietroglyph/walk@v0.0.0-20180728074727-d9c93ed8784b/textedit.go:147: func (te *TextEdit) AppendText(value string) {
github.com/pietroglyph/walk@v0.0.0-20180728074727-d9c93ed8784b/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/pirogom/walk@v0.0.0-20230128105705-492944454fc3/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/planetscale/log@v0.0.0-20230320220358-9d4ed51abd8d/encoder.go:195: func (enc *prettyEncoder) AppendBinary(value []byte) {
github.com/pritunl/mongo-go-driver@v1.11.3/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
github.com/pritunl/mongo-go-driver@v1.11.3/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
github.com/pwiecz/atk@v0.0.0-20210227100443-cb0bf65e43e2/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/pzartem/libxml2@v0.0.0-20200831082948-c3d9892b3a0f/dom/node_element.go:62: func (n *Element) AppendText(s string) error {
github.com/qsirwyk/walk@v1.1.1/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/qsirwyk/walk@v1.1.1/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/raceresult/walk@v0.0.4/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/readykit/gd@v0.0.0-20221015225739-25a193a18dd7/api.go:25532: func (gdClass RichTextLabel) AppendText(bbcode string)  { methodCall[struct{}](gdClass.obj.get(), methodRichTextLabel[76], &bbcode) }
github.com/reusee/ggir@v0.0.0-20141013104036-88ce4855c267/gtk/gtk_traits.go:5592: func (self *TraitComboBoxText) AppendText(text string) {
github.com/richardwilkes/cef@v0.5.3/cef/Textfield_gen.go:111: func (d *Textfield) AppendText(text string) {
github.com/romychs/gotk3@v0.0.0-20210328092339-116e09e25f5e/gtk/combobox.go:240: func (v *ComboBoxText) AppendText(text string) {
github.com/rstshardware/walk@v0.0.1/textedit.go:241: func (te *TextEdit) AppendText(value string) {
github.com/rstshardware/walk@v0.0.1/examples/logview/logview.go:53: func (lv *LogView) AppendText(value string) {
github.com/scjtqs2/walk@v0.0.0-20211230061331-3545c43f16e8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/scjtqs2/walk@v0.0.0-20211230061331-3545c43f16e8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_c.go:1059: func (c *ComboBox) AppendText(text string) { ComboBoxAppendText(c, text) }
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_c.go:1109: func (c *ComboBoxText) AppendText(text string) { ComboBoxTextAppendText(c, text) }
github.com/tHinqa/outside-gtk2@v0.0.0-20131129203352-f05e200c0477/gtk/gtk_e.go:194: func (e *Entry) AppendText(text string)            { EntryAppendText(e, text) }
github.com/tailscale/walk@v0.0.0-20230502163740-ea8725bca6d8/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/tailscale/walk@v0.0.0-20230502163740-ea8725bca6d8/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/tiborvass/uniline@v0.0.0-20150528164013-b39ee5b15151/utils.go:58: func (t text) AppendText(n text) text {
github.com/timelessnesses/l4d2-fmsa-tk@v0.0.0-20230302131554-ce07a2f78b70/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/tmaxmax/go-sse@v0.4.2/message.go:98: func (e *Message) AppendText(chunks ...string) {
github.com/turutcrane/cefingo@v0.5.9/capi/cefingo_gen.go:27863: func (self *CTextfieldT) AppendText(
github.com/vikulin/walk@v0.0.0-20210329133517-4f03df2e225a/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/vikulin/walk@v0.0.0-20210329133517-4f03df2e225a/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/visualfc/atk@v1.2.3/tk/text.go:410: func (w *Text) AppendText(text string) error {
github.com/wangch/walk@v0.0.0-20150802023425-cdcaffe9ed2d/textedit.go:94: func (te *TextEdit) AppendText(value string) {
github.com/wangch/walk@v0.0.0-20150802023425-cdcaffe9ed2d/examples/logview/logview.go:63: func (lv *LogView) AppendText(value string) {
github.com/wayf-dk/go-libxml2@v0.0.0-20210718205101-95691c57c1ac/dom/node_element.go:65: func (n *Element) AppendText(s string) error {
github.com/weili71/go-filex@v0.0.0-20230321132739-7a825b3f5691/filex.go:200: func (f *Filex) AppendText(file *os.File, text string) (int, error) {
github.com/whtiehack/wingui@v0.0.6/edit.go:50: func (e *Edit) AppendText(value string) {
github.com/winxxp/walk@v0.0.0-20200901104446-3e7ddcbafc7d/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/winxxp/walk@v0.0.0-20200901104446-3e7ddcbafc7d/examples/logview/logview.go:55: func (lv *LogView) AppendText(value string) {
github.com/xlplbo/walk@v0.0.0-20201127063646-ea1cc4883b4d/textedit.go:240: func (te *TextEdit) AppendText(value string) {
github.com/xlplbo/walk@v0.0.0-20201127063646-ea1cc4883b4d/examples/logview/logview.go:38: func (lv *LogView) AppendText(value string) {
github.com/yamnikov-oleg/go-gtk@v0.0.0-20160312170510-bc91c0fbef0c/gtk/gtk.go:5343: func (v *ComboBox) AppendText(text string) {
github.com/yamnikov-oleg/go-gtk@v0.0.0-20160312170510-bc91c0fbef0c/gtk/gtk.go:5430: func (v *ComboBoxText) AppendText(text string) {
github.com/zhangweiii/pholcus@v0.0.0-20200228101341-71bf9bad259b/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
github.com/zhangyiming748/sendEmailAlert@v0.0.14/alert.go:67: func (i *Info) AppendText(s string) {
github.com/zurek87/go-gtk3@v0.0.0-20171203223943-bec742635bb7/gtk3/gtk3.go:5867: func (v *ComboBoxText) AppendText(text string) {
github.com/zxjsdp/specimen-go@v1.9.4/gui/windows/logview.go:63: func (lv *LogView) AppendText(value string) {
gitlab.com/abibino-lab/mobal@v0.0.0-20170907170701-2d91a9572e70/gui/logview.go:59: func (lv *LogView) AppendText(value string) {
gitlab.com/gitlab-org/security-products/analyzers/common/table@v0.0.0-20181219150649-131c639c2176/table.go:30: func (t *Table) AppendText(text string) {
gitlab.com/gitlab-org/security-products/analyzers/common/table/v2@v2.0.0-20201123204353-3743d5d8b4cc/table.go:30: func (t *Table) AppendText(text string) {
go.mongodb.org/mongo-driver@v1.12.0/x/bsonx/bsoncore/bson_arraybuilder.go:94: func (a *ArrayBuilder) AppendBinary(subtype byte, b []byte) *ArrayBuilder {
go.mongodb.org/mongo-driver@v1.12.0/x/bsonx/bsoncore/bson_documentbuilder.go:82: func (db *DocumentBuilder) AppendBinary(key string, subtype byte, b []byte) *DocumentBuilder {
modernc.org/atk@v1.4.3/text.go:410: func (w *Text) AppendText(text string) error {

@mateusz834
Copy link
Member

This is motivated by netip.Addr.MarshalText being a non-trivial amount of slowdown in a large JSON object.

Just so you now there is netip.Addr.AppendTo, that does the same thing as MarshalText, but appends.

@dsnet
Copy link
Member Author

dsnet commented Aug 31, 2023

There are unfortunately two problems with AppendTo:

  • It doesn't return an error, and so can't implement equivalent semantics as MarshalText
  • It's naming is ambiguous with regard to whether it does "binary" or "textual" marshaling

@mateusz834
Copy link
Member

@dsnet Yes I agree, just wanted to point that there will be now two methods that do the same thing on netip.Addr, probably after this proposal gets accepted the AppendTo should be deprecated.

@mvdan
Copy link
Member

mvdan commented Aug 31, 2023

Scratching my head and wondering how this hasn't been proposed before :) sounds great to me.

@flimzy
Copy link
Contributor

flimzy commented Aug 31, 2023 •
edited
Loading

What are the possible security implications wrt:

// Implementations must not retain b, nor mutate any bytes within b[:len(b)].

I can imagine a malicious, or sloppy, library not following this rule, and introducing nastiness.

Given the other vectors for "nastiness" that already exist throughout the stdlib when using untrusted code, probably not a meaningful consideration, but it does seem a bit icky to me, to expose some other code's data to arbitrary code this way.

@dsnet
Copy link
Member Author

dsnet commented Aug 31, 2023

Given the other vectors for "nastiness" that already exist throughout the stdlib when using untrusted code, probably not a meaningful consideration, but it does seem a bit icky to me, to expose some other code's data to arbitrary code this way.

Agreed. That's the strength of the MarshalText API. It's much harder to hold wrong.

That said, two of the top-three interfaces in Go by far are io.Reader and io.Writer, both of which expose a buffer to the implementation and document:

Implementations must not retain p.

I don't think we're making the situation any worse since encoding.AppendText will be multiple orders of magnitude less popular than io.Reader or io.Writer.

@seankhliao seankhliao mentioned this issue Sep 1, 2023
Closed
@seebs
Copy link
Contributor

seebs commented Sep 1, 2023

This seems very desireable, the large allocations can be a problem. Although I think in a lot of the cases that are particularly messy, append might not help, because i really need to do an encoder to an io.Writer? But I would still very much like to have this, because iterative loops that are marshaling a bunch of things but only need one at a time are pretty common.

@mateusz834
Copy link
Member

@seebs That is true that something like this would be a nice alternative.

type TextWriter interface {
    WriteText(b io.Writer) error
}

But, this as as side effect would cause another bunch of memory allocations inside each implementation of WriteText, because escape analysis is not great for this use case.

@dsnet
Copy link
Member Author

dsnet commented Sep 1, 2023 •
edited
Loading

Any encoder writing eventually to an io.Writer almost certainly have an intermediate buffer on hand that it can pass to AppendText and then pass eventually to io.Writer.Write.

@yerden
Copy link

yerden commented Sep 22, 2023

Upvoting. I cannot imagine any heavily loaded application tinkering with bytes and text that wouldn't implement an interface of similar semantics.

@mitar
Copy link
Contributor

mitar commented Oct 6, 2023

nor mutate any bytes within b[:len(b)].

Why, performance wise, would that not be allowed? Maybe we can open additional use cases if we do allow that?

@dsnet
Copy link
Member Author

dsnet commented Oct 6, 2023

Why, performance wise, would that not be allowed?

For a function named "Append", it just seems highly suspicious for it to mutate data within b[:len(b)]. The verb "append" implies only modifying data within b[len(b):cap(b)].

A performance argument could be made in the reverse. The caller of AppendText could in theory spin off a goroutine to do some form of read-only validation within b[:len(b)]. Mutating that section would result in a race condition.

@mitar
Copy link
Contributor

mitar commented Oct 6, 2023

For a function named "Append", it just seems highly suspicious for it to mutate data within b[:len(b)]. The verb "append" implies only modifying data within b[len(b):cap(b)].

So should we rename it then? Like MutateText(b []byte) ([]byte, error)?

A performance argument could be made in the reverse. The caller of AppendText could in theory spin off a goroutine to do some form of read-only validation within b[:len(b)]. Mutating that section would result in a race condition.

I think this is handled by the "Implementations must not retain b", if you make a goroutine, you are retaining b.

@dsnet
Copy link
Member Author

dsnet commented Oct 6, 2023

So should we rename it then?

I'm arguing that we should keep it named AppendText and also keep the documentation that implementations shouldn't mutate b[:len(b)].

I think this is handled by the "Implementations must not retain b", if you make a goroutine, you are retaining b.

In my example, the asynchronous validation logic is done by the caller of AppendText, not by the implementation of AppendText.

@mitar
Copy link
Contributor

mitar commented Oct 7, 2023

@dsnet Thanks for the explanation!

@cristaloleg cristaloleg mentioned this issue Dec 16, 2023
Merged
@mvdan
Copy link
Member

mvdan commented Apr 2, 2024

@ianlancetaylor could I nudge this proposal forward? It would help with the encoding/json/v2 efforts, and it seems to have reasonable consensus :)

@icholy icholy mentioned this issue Apr 23, 2024
Open
@rsc
Copy link
Contributor

rsc commented Apr 24, 2024

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals Apr 24, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/605758 mentions this issue: math/big,regexp: implement the encoding.TextAppender interface

apocelipes added a commit to apocelipes/go that referenced this issue Aug 15, 2024
@apocelipes
math/big,regexp: implement the encoding.TextAppender interface
d535941 
For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 15, 2024
@apocelipes
math/big,regexp: implement the encoding.TextAppender interface
f1b2710 
For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 15, 2024
@apocelipes
math/big,regexp: implement the encoding.TextAppender interface
f063a0c 
For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 15, 2024
@apocelipes
math/big,regexp: implement the encoding.TextAppender interface
c258d18 
For golang#62384
gopherbot pushed a commit that referenced this issue Aug 15, 2024
@apocelipes @gopherbot
math/big,regexp: implement the encoding.TextAppender interface
5276107 
For #62384

Change-Id: I1557704c6a0f9c6f3b9aad001374dd5cdbc99065
GitHub-Last-Rev: c258d18
GitHub-Pull-Request: #68893
Reviewed-on: https://go-review.googlesource.com/c/go/+/605758
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
@apocelipes
time: implement the encoding.(Binary|Text)Appender for Time
fc953c8 
"Time.MarshalText" could also gain some performance improvements:

goos: linux
goarch: amd64
pkg: time
cpu: Intel(R) Core(TM) i5-10200H CPU @ 2.40GHz
              │     old      │                 new                 │
              │    sec/op    │   sec/op     vs base                │
MarshalText-8   103.20n ± 1%   67.41n ± 1%  -34.68% (p=0.000 n=10)

              │    old     │                new                 │
              │    B/op    │   B/op     vs base                 │
MarshalText-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

              │    old     │                 new                 │
              │ allocs/op  │ allocs/op   vs base                 │
MarshalText-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
@apocelipes apocelipes mentioned this issue Aug 19, 2024
Closed
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/606655 mentions this issue: time: implement the encoding.(Binary|Text)Appender for Time

apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
@apocelipes
time: implement the encoding.(Binary|Text)Appender for Time
3dc72f4 
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
@apocelipes
time: implement the encoding.(Binary|Text)Appender for Time
7773415 
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 19, 2024
@apocelipes
time: implement the encoding.(Binary|Text)Appender for Time
e04f8df 
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
gopherbot pushed a commit that referenced this issue Aug 20, 2024
@apocelipes @gopherbot
time: implement the encoding.(Binary|Text)Appender for Time
819b1b4 
"Time.Marshal(Binary|Text)" could also gain some performance
improvements. Here is the benchmark highlight:

                │     old      │                 new                 │
                │    sec/op    │   sec/op     vs base                │
MarshalText-8     104.00n ± 3%   67.27n ± 2%  -35.32% (p=0.000 n=10)
MarshalBinary-8    31.77n ± 2%   12.13n ± 1%  -61.82% (p=0.000 n=10)
geomean            57.48n        28.57n       -50.30%

                │    old     │                  new                   │
                │    B/op    │   B/op     vs base                     │
MarshalText-8     48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   16.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                │    old     │                   new                   │
                │ allocs/op  │ allocs/op   vs base                     │
MarshalText-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
MarshalBinary-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For #62384

Change-Id: I320421878a341abf8d668fd57b27292cdfa61330
GitHub-Last-Rev: e04f8df
GitHub-Pull-Request: #68942
Reviewed-on: https://go-review.googlesource.com/c/go/+/606655
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
apocelipes added a commit to apocelipes/go that referenced this issue Aug 21, 2024
@apocelipes
crypto/x509,math/rand/v2: implement the encoding.(Binary|Text)Appender
78abf9c 
Implement the encoding.(Binary|Text)Appender interfaces for "x509.OID".

Implement the encoding.BinaryAppender interface for "rand/v2.PCG" and
"rand/v2.ChaCha8".

"rand/v2.ChaCha8.MarshalBinary" alse gains some performance benefits:

                           │     old      │                 new                 │
                           │    sec/op    │   sec/op     vs base                │
ChaCha8MarshalBinary-8       33.730n ± 2%   9.786n ± 1%  -70.99% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8    99.86n ± 1%   17.79n ± 0%  -82.18% (p=0.000 n=10)
geomean                       58.04n        13.19n       -77.27%

                           │    old     │                  new                   │
                           │    B/op    │   B/op     vs base                     │
ChaCha8MarshalBinary-8       48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   83.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                           │    old     │                   new                   │
                           │ allocs/op  │ allocs/op   vs base                     │
ChaCha8MarshalBinary-8       1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For golang#62384
@apocelipes apocelipes mentioned this issue Aug 21, 2024
Closed
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/607079 mentions this issue: crypto/x509,math/rand/v2: implement the encoding.(Binary|Text)Appender

gopherbot pushed a commit that referenced this issue Aug 21, 2024
@apocelipes @gopherbot
crypto/x509,math/rand/v2: implement the encoding.(Binary|Text)Appender
fd985d2 
Implement the encoding.(Binary|Text)Appender interfaces for "x509.OID".

Implement the encoding.BinaryAppender interface for "rand/v2.PCG" and "rand/v2.ChaCha8".

"rand/v2.ChaCha8.MarshalBinary" alse gains some performance benefits:

                           │     old      │                 new                 │
                           │    sec/op    │   sec/op     vs base                │
ChaCha8MarshalBinary-8       33.730n ± 2%   9.786n ± 1%  -70.99% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8    99.86n ± 1%   17.79n ± 0%  -82.18% (p=0.000 n=10)
geomean                       58.04n        13.19n       -77.27%

                           │    old     │                  new                   │
                           │    B/op    │   B/op     vs base                     │
ChaCha8MarshalBinary-8       48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   83.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                           │    old     │                   new                   │
                           │ allocs/op  │ allocs/op   vs base                     │
ChaCha8MarshalBinary-8       1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
ChaCha8MarshalBinaryRead-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

For #62384

Change-Id: I604bde6dad90a916012909c7260f4bb06dcf5c0a
GitHub-Last-Rev: 78abf9c
GitHub-Pull-Request: #68987
Reviewed-on: https://go-review.googlesource.com/c/go/+/607079
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
4ebb58e 
Implement the encoding.TextAppender for "net.IP".

Implament the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
@apocelipes apocelipes mentioned this issue Aug 22, 2024
Closed
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/607520 mentions this issue: net,net/netip: implement the encoding.(Binary|Text)Appender

apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
75e6f0d 
Implement the encoding.TextAppender for "net.IP".

Implament the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
ac1d994 
Implement the encoding.TextAppender interface for "net.IP".

Implament the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
7c3dd9b 
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Aug 22, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
3268c0d 
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
@nvanbenschoten nvanbenschoten mentioned this issue Aug 26, 2024
Open
apocelipes added a commit to apocelipes/go that referenced this issue Sep 3, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
2e7adac 
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Sep 19, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
e66bcef 
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                     │     old      │                 new                 │
                     │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8    68.35n ± 1%   14.64n ± 1%  -78.57% (p=0.000 n=10)
IPMarshalText/IPv6-8   124.35n ± 2%   62.73n ± 1%  -49.55% (p=0.000 n=10)
geomean                 92.19n        30.31n       -67.12%

                     │    old     │                  new                   │
                     │    B/op    │   B/op     vs base                     │
IPMarshalText/IPv4-8   32.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   48.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=10)

                     │    old     │                   new                   │
                     │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
apocelipes added a commit to apocelipes/go that referenced this issue Sep 20, 2024
@apocelipes
net,net/netip: implement the encoding.(Binary|Text)Appender
5d27854 
Implement the encoding.TextAppender interface for "net.IP".

Implement the encoding.(Binary|Text)Appender interfaces for
"netip.Addr", "netip.AddrPort" and "netip.Prefix".

"net.IP.MarshalText" gets some performance improvements:

                          │     old      │                 new                 │
                          │    sec/op    │   sec/op     vs base                │
IPMarshalText/IPv4-8         66.06n ± 1%   14.55n ± 1%  -77.97% (p=0.000 n=10)
IPMarshalText/IPv6-8        117.00n ± 1%   63.18n ± 1%  -46.00% (p=0.000 n=10)
IPMarshalText/IPv6_long-8    137.8n ± 1%   111.3n ± 1%  -19.27% (p=0.000 n=10)
geomean                      102.1n        46.77n       -54.21%

                          │    old     │                   new                   │
                          │    B/op    │    B/op     vs base                     │
IPMarshalText/IPv4-8        32.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8        48.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6_long-8   96.00 ± 0%   48.00 ± 0%   -50.00% (p=0.000 n=10)

                          │    old     │                   new                   │
                          │ allocs/op  │ allocs/op   vs base                     │
IPMarshalText/IPv4-8        2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6-8        2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
IPMarshalText/IPv6_long-8   2.000 ± 0%   1.000 ± 0%   -50.00% (p=0.000 n=10)

All exported types in the standard library that implement the
"encoding.(Binary|Text)Marshaler" now also implement the
"encoding.(Binary|Text)Appender".

Fixes golang#62384
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Performance Proposal Proposal-Accepted
Projects
Proposals
Status: Accepted
Milestone
Backlog
Development

Successfully merging a pull request may close this issue.

net,net/netip: implement the encoding.(Binary|Text)Appender apocelipes/go
9 participants
@rsc @seebs @mitar @mvdan @dsnet @yerden @flimzy @gopherbot @mateusz834