encoding/xml: Unmarshalling empty XML attribute to int throws error #19333

neocortical · 2017-03-01T00:05:30Z

Go version:
1.8 (regression from 1.7.x --> 1.8.0)

Environment:
Dockerized Alpine (golang:1.8-alpine and golang:1.7-alpine)

Description:
In Go 1.7, an XML entity like <Object foo=""></Object> could be unmarshalled into a struct where Foo is an int value without throwing an error. When upgrading to Go 1.8.0, the same code throws an error due to strconv.ParseInt being called with the empty string as an argument.

This seems to be a behavioral regression between Go 1.7.x and Go 1.8.0.

Example code:
https://play.golang.org/p/GIsONzXQQQ

Expected behavior:
Example code should return without error, printing Foo: 0

Actual behavior:
An error is thrown, indicating that ParseInt was called with an empty string.

Compiling this same code with Go 1.7.x results in the expected behavior.

The text was updated successfully, but these errors were encountered:

tobbbles · 2017-03-06T13:37:14Z

Recently tried upgrading to 1.8 and ran into this.

Many of our XML-based 3rd parties leave attributes empty for prices, price="". I think unmarshalling to the type default is expected here.

SamWhited · 2017-03-09T21:56:39Z

This appears to have been introduced by 2c58cb3 which returns an error that was previously ignored.

This is a wider change that may have affected several types; I'm unsure if it breaks the compatibility promise, or was fixing a bug. I'm inclined to think that it doesn't make sense for ints to unmarshal to their zero value in XML, but this is one of those weird cases where Go types/XML aren't really compatible and any mapping between them will have to do some funky stuff, and if it's always been doing this maybe we should special case it and ignore the err result of strconv.

/cc @ericlagergren (author), @bradfitz (reviewer)

bradfitz · 2017-03-09T22:01:20Z

@rsc, do you have opinions here?

SamWhited · 2017-03-09T22:18:43Z

Also worth noting: before that commit unmarshaling values did return an strconv error, it was only unmarshaling attributes that ignored it. Test:

// Issue 19333. Unmarshaling empty attr or element into int must not error.
func TestUnmarsdhalInt(t *testing.T) {
	t.Run("Attr", func(t *testing.T) {
		v := &struct {
			XMLName Name `xml:"int"`
			Foo     int  `xml:"foo,attr"`
		}{}
		if err := Unmarshal([]byte(`<int foo=""></int>`), v); err != nil {
			t.Errorf("did not expect error, but got `%s`", err)
		}
		if v.Foo != 0 {
			t.Errorf("want 0, have %d", v.Foo)
		}
	})
	t.Run("Value", func(t *testing.T) {
		v := &struct {
			XMLName Name `xml:"int"`
			Foo     int  `xml:"foo"`
		}{}
		if err := Unmarshal([]byte(`<int><foo></foo></int>`), v); err != nil {
			t.Errorf("did not expect error, but got `%s`", err)
		}
		if v.Foo != 0 {
			t.Errorf("want 0, have %d", v.Foo)
		}
	})
}

ericlagergren · 2017-03-09T22:31:35Z

Like @SamWhited said, it makes the most sense to special case attr="" as the Go type's zero value.

I created the original issue because I tried to marshal a single letter into a byte and instead of returning an error saying, "you can't do that" it just silently left the field as 0.

The alternative (reverting 2c58cb3) means if the client sends an invalid XML request (e.g., attr="foobar" into an int64) the server has no way to determine whether it was a bad request or the field was meant to be 0.

I believe that behavior is definitely a bug in the XML package.

s-mang · 2017-03-22T22:13:14Z

related to #13417
and my open CL https://go-review.googlesource.com/c/38386

s-mang · 2017-03-23T00:03:11Z

Fixed via 0a0186f

rsc · 2017-04-05T14:15:49Z

Reopening for backport I guess.

rsc · 2017-04-05T14:17:48Z

I can't figure out what is going on here. The linked commit 0a0186f changes the behavior of the XML package beyond this bug fix. It is not appropriate for Go 1.8.1. Leaving this open because we need to decide whether to live with the bug or prepare a different release-specific fix.

SamWhited · 2017-04-05T14:21:14Z

Could we revert 2c58cb3 for 1.8.1 only?

ericlagergren · 2017-04-05T14:25:19Z

Russ—I think the linked commit only happened to fix this issue and was in response to another bug. See Sarah Adams' comments.

…

On Wed, Apr 5, 2017 at 7:22 AM Sam Whited ***@***.***> wrote: Could we revert 2c58cb3 <2c58cb3> for 1.8.1 only? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19333 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFnwZ9PBKXM9qfwHhLsA_k1DPtfG6JVpks5rs6OKgaJpZM4MPDYh> .

rsc · 2017-04-05T15:37:14Z

OK but the bug was marked Go 1.8.1 and closed with a link to that commit. That commit is an inappropriate fix for Go 1.8.1 (and not cherry-picked yet anyway), so reopening.

rsc · 2017-04-05T15:48:42Z

@SamWhited, yes, that seems to be the solution. I will send a CL.

s-mang · 2017-04-05T16:05:47Z

My apologies for closing.

rsc · 2017-04-05T16:06:22Z

No worries @adams-sarah, it's all very confusing around point release milestones. We hope to have a bot help with this soon.

rsc · 2017-04-05T16:08:54Z

Here is a test program. It covers the cases in this issue (attributes) and the ones in #13417 (elements):

package main

import (
	"encoding/xml"
	"fmt"
)

func main() {
	type X struct {
		XMLName xml.Name `xml:"X"`
		A       int      `xml:",attr"`
		C       int
	}

	var tests = []struct {
		input string
		ok    bool
	}{
		{`<X></X>`, true},
		{`<X A=""></X>`, true},
		{`<X A="bad"></X>`, false},
		{`<X></X>`, true},
		{`<X><C></C></X>`, true},
		{`<X><C/></X>`, true},
		{`<X><C>bad</C></X>`, false},
	}

	for _, tt := range tests {
		err := xml.Unmarshal([]byte(tt.input), new(X))
		if err != nil {
			fmt.Printf("%-20s ERROR %v\n", tt.input, err)
		} else {
			fmt.Printf("%-20s ok\n", tt.input)
		}
	}
}

Go 1.2 through Go 1.7 were consistent: attributes unchecked, children strictly checked:

$ go1.7 run /tmp/x.go
<X></X>              ok
<X A=""></X>         ok
<X A="bad"></X>      ok
<X></X>              ok
<X><C></C></X>       ERROR strconv.ParseInt: parsing "": invalid syntax
<X><C/></X>          ERROR strconv.ParseInt: parsing "": invalid syntax
<X><C>bad</C></X>    ERROR strconv.ParseInt: parsing "bad": invalid syntax
$

Go 1.8 made attributes strictly checked, matching children:

$ go1.8 run /tmp/x.go
<X></X>              ok
<X A=""></X>         ERROR strconv.ParseInt: parsing "": invalid syntax
<X A="bad"></X>      ERROR strconv.ParseInt: parsing "bad": invalid syntax
<X></X>              ok
<X><C></C></X>       ERROR strconv.ParseInt: parsing "": invalid syntax
<X><C/></X>          ERROR strconv.ParseInt: parsing "": invalid syntax
<X><C>bad</C></X>    ERROR strconv.ParseInt: parsing "bad": invalid syntax
$

Go 1.9 will (at least right now) relax things so that only non-empty bad inputs are checked:

$ go run /tmp/x.go  # Go 1.9 development
<X></X>              ok
<X A=""></X>         ok
<X A="bad"></X>      ERROR strconv.ParseInt: parsing "bad": invalid syntax
<X></X>              ok
<X><C></C></X>       ok
<X><C/></X>          ok
<X><C>bad</C></X>    ERROR strconv.ParseInt: parsing "bad": invalid syntax
$

For Go 1.8.1, I will revert the checking behavior back to Go 1.7 for now. We can try another attempt at attribute checking, with @adams-sarah's strictness relaxation applied both to attributes and children, in Go 1.9.

$ go run /tmp/x.go  # Go 1.8.1 development
<X></X>              ok
<X A=""></X>         ok
<X A="bad"></X>      ok
<X></X>              ok
<X><C></C></X>       ERROR strconv.ParseInt: parsing "": invalid syntax
<X><C/></X>          ERROR strconv.ParseInt: parsing "": invalid syntax
<X><C>bad</C></X>    ERROR strconv.ParseInt: parsing "bad": invalid syntax

gopherbot · 2017-04-05T16:30:20Z

CL https://golang.org/cl/39607 mentions this issue.

rsc · 2017-04-05T17:00:33Z

Cherry-picked.

…ntax, like Go 1.7 Consider this struct, which expects an attribute A and a child C both ints: type X struct { XMLName xml.Name `xml:"X"` A int `xml:",attr"` C int } Go 1.2 through Go 1.7 were consistent: attributes unchecked, children strictly checked: $ go1.7 run /tmp/x.go <X></X> ok <X A=""></X> ok <X A="bad"></X> ok <X></X> ok <X><C></C></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X><C/></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X><C>bad</C></X> ERROR strconv.ParseInt: parsing "bad": invalid syntax $ Go 1.8 made attributes strictly checked, matching children: $ go1.8 run /tmp/x.go <X></X> ok <X A=""></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X A="bad"></X> ERROR strconv.ParseInt: parsing "bad": invalid syntax <X></X> ok <X><C></C></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X><C/></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X><C>bad</C></X> ERROR strconv.ParseInt: parsing "bad": invalid syntax $ but this broke XML code that had empty attributes (#19333). In Go 1.9 we plan to start allowing empty children (#13417). The fix for that will also make empty attributes work again: $ go run /tmp/x.go # Go 1.9 development <X></X> ok <X A=""></X> ok <X A="bad"></X> ERROR strconv.ParseInt: parsing "bad": invalid syntax <X></X> ok <X><C></C></X> ok <X><C/></X> ok <X><C>bad</C></X> ERROR strconv.ParseInt: parsing "bad": invalid syntax $ For Go 1.8.1, we want to restore the empty attribute behavior to match Go 1.7 but not yet change the child behavior as planned for Go 1.9, since that change hasn't been through release testing. Instead, restore the more lax Go 1.7 behavior, so that XML files with empty attributes will not be broken until Go 1.9: $ go run /tmp/x.go # after this CL <X></X> ok <X A=""></X> ok <X A="bad"></X> ok <X></X> ok <X><C></C></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X><C/></X> ERROR strconv.ParseInt: parsing "": invalid syntax <X><C>bad</C></X> ERROR strconv.ParseInt: parsing "bad": invalid syntax $ Fixes #19333. Change-Id: I3d38ebd2509f5b6ea3fd4856327f887f9a1a8085 Reviewed-on: https://go-review.googlesource.com/39607 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Sarah Adams <shadams@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>

bradfitz added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Mar 9, 2017

bradfitz assigned rsc Mar 9, 2017

bradfitz added this to the Go1.8.1 milestone Mar 21, 2017

s-mang closed this as completed Mar 23, 2017

rsc reopened this Apr 5, 2017

rsc closed this as completed Apr 5, 2017

shogo82148 mentioned this issue May 16, 2017

support Go1.8, and drop support Go1.1 shogo82148/androidbinary#5

Merged

ulisesflynn mentioned this issue Jul 21, 2017

Replace *NullBool/*NullInt64 with *bool/*int and bool/int braintree-go/braintree-go#176

Merged

golang locked and limited conversation to collaborators Apr 5, 2018

gopherbot added the FrozenDueToAge label Apr 5, 2018

rsc removed their assignment Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

encoding/xml: Unmarshalling empty XML attribute to int throws error #19333

encoding/xml: Unmarshalling empty XML attribute to int throws error #19333

neocortical commented Mar 1, 2017 •

edited

tobbbles commented Mar 6, 2017

SamWhited commented Mar 9, 2017 •

edited

bradfitz commented Mar 9, 2017

SamWhited commented Mar 9, 2017 •

edited

ericlagergren commented Mar 9, 2017 •

edited

s-mang commented Mar 22, 2017

s-mang commented Mar 23, 2017

rsc commented Apr 5, 2017

rsc commented Apr 5, 2017

SamWhited commented Apr 5, 2017 •

edited

ericlagergren commented Apr 5, 2017 via email •

edited

rsc commented Apr 5, 2017

rsc commented Apr 5, 2017

s-mang commented Apr 5, 2017

rsc commented Apr 5, 2017

rsc commented Apr 5, 2017

gopherbot commented Apr 5, 2017

rsc commented Apr 5, 2017

encoding/xml: Unmarshalling empty XML attribute to int throws error #19333

encoding/xml: Unmarshalling empty XML attribute to int throws error #19333

Comments

neocortical commented Mar 1, 2017 • edited

tobbbles commented Mar 6, 2017

SamWhited commented Mar 9, 2017 • edited

bradfitz commented Mar 9, 2017

SamWhited commented Mar 9, 2017 • edited

ericlagergren commented Mar 9, 2017 • edited

s-mang commented Mar 22, 2017

s-mang commented Mar 23, 2017

rsc commented Apr 5, 2017

rsc commented Apr 5, 2017

SamWhited commented Apr 5, 2017 • edited

ericlagergren commented Apr 5, 2017 via email • edited

rsc commented Apr 5, 2017

rsc commented Apr 5, 2017

s-mang commented Apr 5, 2017

rsc commented Apr 5, 2017

rsc commented Apr 5, 2017

gopherbot commented Apr 5, 2017

rsc commented Apr 5, 2017

neocortical commented Mar 1, 2017 •

edited

SamWhited commented Mar 9, 2017 •

edited

SamWhited commented Mar 9, 2017 •

edited

ericlagergren commented Mar 9, 2017 •

edited

SamWhited commented Apr 5, 2017 •

edited

ericlagergren commented Apr 5, 2017 via email •

edited