proposal: encoding/xml: allow caller to specify preferred namespace prefix

[ydnar]

What version of Go are you using (go version)?

$ go version
go version go1.17.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ydnar/Library/Caches/go-build"
GOENV="/Users/ydnar/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/ydnar/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ydnar/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.17.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.17.1/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/ydnar/development/ydnar/go/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6g/1gng3zts0t39s_qbtt7p0wsc0000gn/T/go-build2847614851=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Use xml.Marshal to marshal namespaced XML, and xml.Unmarshal to decode that XML.

Example: https://play.golang.org/p/-6Ee8tcLl2L

What did you expect to see?

Namespace prefixes preserved round-trip.

What did you see instead?

Namespace prefixes written, but not recognized as the relevant tag when decoding.

Proposal

Change the encoding/xml package to allow callers to specify a preferred namespace prefix, and allow decoding of prefixed tags.

The preferred prefix can be specified by prefixing the XML tag name in a struct tag or (xml.Name).Local.

This should be backwards-compatible with existing Go 1 encoding/xml clients.

Example

type EPP struct {
	XMLName struct{} `xml:"urn:ietf:params:xml:ns:epp-1.0 epp"`
	Command *Command `xml:"command,omitempty"`
}

type Command struct {
	Check *Check `xml:"urn:ietf:params:xml:ns:epp-1.0 check,omitempty"`
}

type Check struct {
	DomainCheck *DomainCheck `xml:"urn:ietf:params:xml:ns:domain-1.0 domain:check,omitempty"`
}

type DomainCheck struct {
	DomainNames []string `xml:"urn:ietf:params:xml:ns:domain-1.0 domain:name,omitempty"`
}

This change would allow a structure defined above to correctly round-trip encode and decode with namespace prefixes:

var v = &EPP{Command: &Command{Check: &Check{DomainCheck: &DomainCheck{DomainNames: []string{"golang.org", "go.dev"}}}}}

Would encode to (and decode from):

<epp xmlns="urn:ietf:params:xml:ns:epp-1.0"><command><check><domain:check xmlns:domain="urn:ietf:params:xml:ns:domain-1.0"><domain:name>golang.org</domain:name><domain:name>go.dev</domain:name></domain:check></check></command></epp>

Implementation

A working implementation can be found at #48641, built on earlier work by @rogpeppe and others.

Related Issues

This would fix #43168 and update #11431 and #8068.

[tobbee]

@rsc

I think it is really important that the standard library gets proper handling of XML namespaces and namespace prefixes. The current implementation is definitely broken as has been raised in multiple issues. I can elaborate on the problems and short comings if needed,

In my view, this PR provides a good solution, or at least a good basis for a solution. I found myself forced to clone the standard xml package together with this patch in my new repo https://github.com/Eyevinn/dash-mpd that handles DASH MPD XMLs and need proper namespaces and prefixes.

In the discussion about previous ticket, Dave Cheney argued that one cannot change the behavior of such an established stdlib package as XML due to the risk of breaking someones code. I think there is some truth in that argument, so I'd like to suggest that we start working on a variant of the xml package that supports namespaces and namespace prefixes. It could be called encoding/xml2, and this PR could make a good start. It would be nice if it could be done in the 1.21 time frame.

[ydnar]

I’ve rebased #48641 on top of current master: https://go-review.googlesource.com/c/go/+/355353

[ydnar]

Hi there, anything I can do to nudge this along? Thanks!

[ydnar]

Would putting this behind GOEXPERIMENT=xmlns make it more palatable?

[kavu]

Hi! Would you mind I raise this issue again? What's the consensus regarding this one?

[ydnar]

The proposed change is spiritually similar to the recent change to allow HTTP verbs in http.ServeMux patterns: https://pkg.go.dev/net/http#ServeMux

[3052]

this example is not robust:

https://play.golang.org/p/-6Ee8tcLl2L

as it includes no consideration for namespaced attributes, which are also problematic