Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/xml: Panic when unmarshaling specially crafted structs #16497

Open
SamWhited opened this issue Jul 26, 2016 · 7 comments
Open

encoding/xml: Panic when unmarshaling specially crafted structs #16497

SamWhited opened this issue Jul 26, 2016 · 7 comments
Labels
early-in-cycle A change that should be done early in the 3 month dev cycle. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@SamWhited
Copy link
Member

SamWhited commented Jul 26, 2016

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

go version devel +67f799c Tue Jul 26 00:18:42 2016 +0000 linux/amd64

But it also affects the version of 1.6 on the playground at the moment.

  1. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN="/home/sam/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/sam"
GORACE=""
GOROOT="/home/sam/src/go"
GOTOOLDIR="/home/sam/src/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build800221808=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
  1. What did you do?

When attempting to unmarshal XML into a struct that is composed of another struct that has an XMLName field that has a field number greater than or equal to the number of fields in the original struct, finfo.Value panics because reflect tries to get the value of the field in the parent struct at the location of XMLName in the child struct (which has a field index that is out of bounds when applied to the parent struct).

For example, this contrived example panics:

package main

import (
    "encoding/xml"
)

type IQ struct {
    Type    string   `xml:"type,attr"`
    XMLName xml.Name `xml:"iq"`
}

func main() {
    resp := struct {
        IQ
    }{}
    xml.Unmarshal([]byte(`<iq/>`), &resp)
}

(Playground)

but if we move the fields around:

type IQ struct {
    XMLName xml.Name `xml:"iq"`
    Type    string   `xml:"type,attr"`
}

it works (Playground)

@bradfitz bradfitz added this to the Go1.8 milestone Jul 26, 2016
@bradfitz bradfitz added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 26, 2016
@SamWhited
Copy link
Member Author

SamWhited commented Jul 26, 2016

Now that it's not the middle of the night I decided to take a look at this; it appears that when composed fields are used in the parent structs metadata, their indexes are rewritten, but this was not done for xmlname fields (which are handled separately). This is a relatively easy fix (CL).

@gopherbot
Copy link
Contributor

CL https://golang.org/cl/25242 mentions this issue.

@SamWhited
Copy link
Member Author

SamWhited commented Jul 27, 2016

I was going to post this on the CL, but I think GitHub or the dev list might be a better place (I never know where to post things). If this is incorrect, please let me know and I'll move it.

Currently the patch I have in the CL breaks two tests, but I think the behavior is wrong and the tests need to be changed. I'm not sure if this behavior is covered by the compatibility promise or not, but XMLNS fixes have been put in place in the past that changed the behavior of marshaling/unmarshaling with the rational that they were bug fixes and not part of the compatibility promise, so I'm hoping this will be simliar.

The issue is that right now if you unmarshal XML into a struct that contains an inner struct with an XMLName the XMLName will be matched against the XML correctly and the struct (and all its attributes) will be unmarshaled correctly however the XMLName field on the inner struct will not be set. If you go the other way and marshal that struct back to XML however the inner XML's XMLName tag will be used and the resulting XML will have the correct name and namespace.

Example — https://play.golang.org/p/bunS8dHMAO

This means that Marshal and Unmarshal are not inverse functions, which feels like a bug to me.

Thoughts? Is it okay to change this behavior so that the example above, instead of printing:

<outerstruct xmlns="urn:example:issue:16497"></outerstruct>
InnerStruct: {XMLName:{Space:urn:example:issue:16497 Local:outerstruct}}
OuterStruct: {InnerStruct:{XMLName:{Space: Local:}}}
OuterOuterStruct: {OuterStruct:{InnerStruct:{XMLName:{Space: Local:}}}}

would print:

<outerstruct xmlns="urn:example:issue:16497"></outerstruct>
InnerStruct: {XMLName:{Space:urn:example:issue:16497 Local:outerstruct}}
OuterStruct: {InnerStruct:{XMLName:{Space:urn:example:issue:16497 Local:outerstruct}}}
OuterOuterStruct: {OuterStruct:{InnerStruct:{XMLName:{Space:urn:example:issue:16497 Local:outerstruct}}}}

@rsc rsc modified the milestones: Go1.9Early, Go1.8 Oct 26, 2016
@bradfitz bradfitz modified the milestones: Go1.10Early, Go1.9Early May 3, 2017
@bradfitz bradfitz added early-in-cycle A change that should be done early in the 3 month dev cycle. and removed early-in-cycle A change that should be done early in the 3 month dev cycle. labels Jun 14, 2017
@bradfitz bradfitz modified the milestones: Go1.10Early, Go1.10 Jun 14, 2017
@rsc rsc modified the milestones: Go1.10, Go1.11 Nov 22, 2017
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/108395 mentions this issue: encoding/xml : fix panic of unmarshaling of anonymous structs

@iwdgo
Copy link
Contributor

iwdgo commented Apr 20, 2018

The crash is caused by known and documented panics of the reflection package. The crash cannot be prevented by fixing the type. The submitted fix checks that no value is requested by anonymous structs which have no value anyway.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/109855 mentions this issue: encoding/xml : Fixes to enforce XML namespace standard

@bradfitz bradfitz modified the milestones: Go1.11, Go1.12 May 18, 2018
@gopherbot gopherbot modified the milestones: Go1.12, Unplanned May 23, 2018
ydnar pushed a commit to ydnar/go that referenced this issue Sep 27, 2021
All issues of 13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml : fix duplication of namespace tags by encoder (7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml : fix unexpected behavior of encoder.Indent("", "") (13185, 11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml : fix overriding by empty namespace (7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml : fix panic on embedded unexported XMLName (10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml : fix panic of unmarshaling of anonymous structs (16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml : fix closing tag failure (20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml : add check of namespaces to detect field names conflicts (8535, 11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml : fix normalization of attributes values (20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (20396)

The occurrence of second : in space and tag name is rejected.

Fixes : golang#7113, golang#7535, golang#8068,  golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165
ydnar pushed a commit to ydnar/go that referenced this issue Sep 27, 2021
All issues of 13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml : fix duplication of namespace tags by encoder (7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml : fix unexpected behavior of encoder.Indent("", "") (13185, 11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml : fix overriding by empty namespace (7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml : fix panic on embedded unexported XMLName (10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml : fix panic of unmarshaling of anonymous structs (16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml : fix closing tag failure (20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml : add check of namespaces to detect field names conflicts (8535, 11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml : fix normalization of attributes values (20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (20396)

The occurrence of second : in space and tag name is rejected.

Fixes : golang#7113, golang#7535, golang#8068,  golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165
ydnar added a commit to ydnar/go that referenced this issue Oct 12, 2021
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests
ydnar added a commit to ydnar/go that referenced this issue Oct 12, 2021
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496
ydnar added a commit to ydnar/go that referenced this issue Oct 14, 2021
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments
ydnar added a commit to ydnar/go that referenced this issue Oct 14, 2021
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments
ydnar added a commit to ydnar/go that referenced this issue Apr 28, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces
ydnar added a commit to ydnar/go that referenced this issue May 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue May 3, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue May 5, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue May 9, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue May 15, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue May 21, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue Jun 1, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue Jul 7, 2023
All issues of golang#13400 which are not new functionalities have fixes.
There are minor incompatibilities between them due to the handling of
prefixes. Duplicating a prefix or an namespace is invalid XML.
This is now avoided. XML produced is always valid.

Tests have been added for each fix and example and previous tests
fixed as output is already more compact is some cases.

encoding/xml: fix duplication of namespace tags by encoder (golang#7535)

A tag prefix identifies the name space of the tag and not the default name
space like xmlns="...". Writing the prefix is incorrect when it is bound
to a name space using the standard xmlns:prefix="..." attribute.
This fix skips this print.

Consequences are
 - duplication is avoided in line with name space standard in reference.
 - the _xmlns declaration does not appear anymore. To keep the previous
behaviour, the prefix is printed in all other cases. Token now always
produces well-formed XML except when the explicit name space collides
with the prefix.

Made prefix handling "xmlns="space" xmlns:_xmlns="xmlns" _..." has
be removed in all wants of tests. In some cases, useless declarations like
xmlns:x="x" are still added in line with previous behavior.

encoding/xml: fix unexpected behavior of encoder.Indent("", "") (golang#13185, golang#11431)

MarshalIndent and Marshal share code. When prefix and indent are
empty, the behavior is like Marshal when it should have a minimal
indent, i.e. new line as in documentation.

A boolean is added to the local printer struct which defaults to false.
It is set to true only when MarshalIndent is used and prefix and indent
are empty.

encoding/xml: fix overriding by empty namespace (golang#7113)

The namespace defined by xmlns="value" can be overridden in
every included tag including by the empty namespace xmlns="".
Empty namespace is not authorized with a prefix (xmlns:ns="").

Method to calculate indent of XML handles depth of tag and
its associated namespace. The fix leaves the method active even
when no indent is required.

An XMLName field in a struct means that namespace must be
enforced even if empty. This occurs only on an inner tag as an
overwrite of any non-empty namespace of its outer tag.

To obtain the xmlns="" required, an attribute is added.

encoding/xml: fix panic on embedded unexported XMLName (golang#10538)

By laws of reflection, unexported fields are unreachable by .Value.
XMLName are allowed at any level of an inner struct but the struct
may not be reachable. If XMLName field is found without name,
it cannot be exported and the value is discarded like other fields.
Some comments have been to underline where the issue arises.
Another XMLName test was incorrectly set up and is fixed.
Various cases added in a specific test.

encoding/xml: fix panic of unmarshaling of anonymous structs (golang#16497)

Encoding/xml is using type "typinfo" which provides its own "value" method
using the reflection package. It fails to check for the documented possible
panics of the reflection methods. It is impossible to fix the method as the
parameter is also using reflection.

The fix is to discard anonymous structs which have no value anyway.
Encoder/xml documentation already mentions that fields are accessed using
the reflection package. A relevant test is added.

encoding/xml: fix closing tag failure (golang#20685)

Push/pop of elements name must be done using the eventual prefix together
with the tag name. The operation is moved before the translation of prefix
into its URI. One end element of a test is fixed as expecting the last used
namespace is incorrect.
After closing a tag using a namespace, the valid namespace must be taken
from the opening tag.

encoding/xml: add check of namespaces to detect field names conflicts (golang#8535, golang#11724)

Comparing namespaces of fields was missing and false conflicts were detected.

encoding/xml: fix invalid empty namespace without prefix (golang#8068)

Empty namespace is allowed only if it has no prefix. An error message is now
returned if a prefix exists. A similar case when no prefix is provided,
thus with syntax xmlns:="..." is also rejected.

encoding/xml: fix normalization of attributes values (golang#20614)

The attribute value was read as text. The existing attribute reader logic is
fixed as an attribute may have a namespace or only a prefix. Other
possibilities have been removed.

To keep the behavior of raw token which allows many faults in attributes list,
error handling is heavily using the Strict parameter of the decoder.
Specific tests have been added including list of attributes.

To keep the behavior or unmarshal, escaped characters handling has been
added but it is not symmetrical to Marshal for quotes but follows XML
specification.

encoding/xml: fix absence of detection of another : in qualified names (golang#20396)

The occurrence of second : in space and tag name is rejected.

Fixes: golang#7113, golang#7535, golang#8068, golang#8535, golang#10538, golang#11431, golang#13185, golang#16497, golang#20396, golang#20614, golang#20685

Change-Id: Ib4a60347a47d23ff59b63307cebb83b71c7c9165

Commit originally authored by: Constantin Konstantinidis <constantinkonstantinidis@gmail.com>

encoding/xml: fix printing of namespace prefix in tag names

Prefix displays in XML when defined in tag names.
The URL of the name space is also returned for an End Token as
documentation requires to improve idempotency of Marshal/Unmarshal.

Translating the prefix is popping the NS which was unavailable
for translation. Translate for an End Token has been moved inside
the pop element part.

Fixes golang#9519

Change-Id: Id7a14d07c106a76a487b5c4e28d9d563fe061c60

encoding/xml: restore changes lost in merge

See golang#43168.

encoding/xml: gofmt

encoding/xml: minimalIndent -> minIndent to shrink diff

encoding/xml: gofmt

encoding/xml: revert changes to (*Decoder).attrval from master

encoding/xml: restore (*printer).writeIndent to master

encoding/xml: remove comments that don’t change functionality of tests

encoding/xml: use t.Errorf instead of fmt.Errorf

encoding/xml: fix test case for golang#11496

encoding/xml: revert unrelated change in (*Decoder).readName

encoding/xml: remove comments

encoding/xml: remove comments

encoding/xml: edit comments

encoding/xml: add test to ensure '@' is legal in an attr val

encoding/xml: improve test error quoting for readability

encoding/xml: (*Decoder).nsname now strictly parses namespaces

encoding/xml: disable 3/4 tests for golang#43168

I’m not sure what the right fix is for this. Malformed namespaces
(leading or trailing colons, more than 1 colon) should result in an error.

encoding/xml: allow callers to specify namespace prefix

A preferred namespace prefix can now be specified by setting the local
name space to "prefix:space".

In a struct tag:

struct Message {
    XMLName struct{} `xml:"urn:ietf:foo foo:message"`
    Value string `xml:"urn:ietf:foo foo:value"`
}

encoding/xml: use tag prefixes if previously defined

Instead of having redundant xmlns= attrs.

encoding/xml: rename (*printer).popPrefix to popPrefixes

since it pops 1 or more prefixes

encoding/xml: rename printer struct fields

To reflect the fact that prefixes apply to tag names, not just attrs.

encoding/xml: url->uri; reorg comments

encoding/xml: fix comments

encoding/xml: elide redundant xmlns= attrs for default namespace (no prefix)

encoding/xml: move namespace prefix tests

encoding/xml: handle namespace prefixes at the root element

encoding/xml: handle namespaced prefixed attr names

encoding/xml: split out prefixes during reflection, not parsing

encoding/xml: rename tagName to xmlTag; document it

encoding/xml: separate printing xmlns:prefix= attr from creating prefix

encoding/xml: clean up comments

encoding/xml: don’t use xmlTag in attr serialization

encoding/xml: test nested prefixed namespaced tags

encoding/xml: add test for prefixed XML tags without xmlns

encoding/xml: update doc comments

encoding/xml: update comments

encoding/xml: preemptively declare and record all namespace prefixes

Immediate children of XML tags that share a namespace and prefix will now
have that namespace prefix declared on the parent tag.

Explicitly declared namespace prefixes via xmlns:prefix="uri" attributes
will now have their values recorded and reused for marshalling child tags.

This commit reorganizes how namespace prefixes are recorded and detected,
along with how start tags are recorded via a slice of element structs.

Some tests are changed, because the value of the marshalled XML has changed,
but the resulting XML should be equivalent (and correct).

encoding/xml: restore tests with xmlns:prefix= attrs to existing behavior

Use xmlnsURL instead of xmlnsPrefix to specify namespace prefixes.

encoding/xml: prefer unprefixed names where possible

encoding/xml: add EncodeToken tests for explicit prefixes

encoding/xml: add test cases for prefixed local names

encoding/xml: revert change in 8f143f5277; do not preemptively declare XML namespaces

encoding/xml: fix typo in test
ydnar added a commit to ydnar/go that referenced this issue Aug 4, 2023