Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fmt: %2x/%02x verb prints one element for a zero length []byte #17024

Closed
kortschak opened this issue Sep 7, 2016 · 12 comments
Closed

fmt: %2x/%02x verb prints one element for a zero length []byte #17024

kortschak opened this issue Sep 7, 2016 · 12 comments

Comments

@kortschak
Copy link
Contributor

@kortschak kortschak commented Sep 7, 2016

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go1.7

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build602491044=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

Run the program at https://play.golang.org/p/p6yew65soY

package main

import (
    "fmt"
)

func main() {
    var b []byte
    fmt.Printf("%q\n", fmt.Sprintf("%02d", b))
    fmt.Printf("%q\n", fmt.Sprintf("%2x", b))
    fmt.Printf("%q\n", fmt.Sprintf("%02x", b))
}

What did you expect to see?

"[]"
""
""

What did you see instead?

"[]"
"  "
"00"
@griesemer
Copy link
Contributor

@griesemer griesemer commented Sep 7, 2016

Not a bug. %02x means that the format is reserving 2 runes of space for the argument, and that that space is 0-filled from the left. Compare with https://play.golang.org/p/QCbJYhHKmz which fills the space with blanks.

@griesemer griesemer closed this Sep 7, 2016
@griesemer
Copy link
Contributor

@griesemer griesemer commented Sep 7, 2016

PS: The '0' in %02x is a format flag, it's not part of the width 2. See https://golang.org/pkg/fmt/ and look for "Other flags" .

@kortschak
Copy link
Contributor Author

@kortschak kortschak commented Sep 7, 2016

That is sort of surprising; the '0' and ' ' are applied per element when the slice is non-zero length (https://play.golang.org/p/8hqqedcxsS), and are not applied at all for %02d in the linked program above. There is a whole heap of non-orthogonality here.

@griesemer
Copy link
Contributor

@griesemer griesemer commented Sep 8, 2016

@kortschak Hm, it is confusing indeed. I see the following sections in the fmt documentation:

For compound operands such as slices and structs, the format applies to the elements of each operand, recursively, not to the operand as a whole. Thus %q will quote each element of a slice of strings, and %6.2f will control formatting for each element of a floating-point array.

However, when printing a byte slice with a string-like verb (%s %q %x %X), it is treated identically to a string, as a single item.

Not sure that's sufficient. Leaving for @robpike to comment.

@robpike
Copy link
Contributor

@robpike robpike commented Sep 8, 2016

Re-opening. It looks like a bug to me. %d is working fine, %x has inconsistencies.

@robpike robpike reopened this Sep 8, 2016
@robpike robpike added this to the Go1.8 milestone Sep 8, 2016
@josharian
Copy link
Contributor

@josharian josharian commented Sep 8, 2016

@kortschak
Copy link
Contributor Author

@kortschak kortschak commented Sep 8, 2016

I just had a look and the change is just the deletion of the condition at https://tip.golang.org/src/fmt/format.go#L357 with associated code readjustment (change the prior condition to be for width == 0 or just for length == 0 before this, and return early). However, that this is specifically in place, and that there are tests for this behaviour suggests someone did intend it.

With the change:

--- FAIL: TestSprintf (0.00s)
        fmt_test.go:1023: Sprintf("%2x", []) = "" want "  "
        fmt_test.go:1023: Sprintf("%#2x", []) = "" want "  "
        fmt_test.go:1023: Sprintf("% 02x", []) = "" want "00"
        fmt_test.go:1023: Sprintf("%# 02x", []) = "" want "00"
        fmt_test.go:1023: Sprintf("%-2x", []) = "" want "  "
        fmt_test.go:1023: Sprintf("%-02x", []) = "" want "  "
        fmt_test.go:1021: Sprintf("%2x", "") = <> want <  >
        fmt_test.go:1021: Sprintf("%#2x", "") = <> want <  >
        fmt_test.go:1021: Sprintf("% 02x", "") = <> want <00>
        fmt_test.go:1021: Sprintf("%# 02x", "") = <> want <00>
        fmt_test.go:1021: Sprintf("%-2x", "") = <> want <  >
        fmt_test.go:1021: Sprintf("%-02x", "") = <> want <  >
@martisch
Copy link
Contributor

@martisch martisch commented Sep 8, 2016

There has been confusion around this in the past.

@robpike added some tests for it with https://go-review.googlesource.com/#/c/11600/
after discussion in #10430 and #11422.

The behavior for padding is consistent with go1.4 and i rewrote the %x formatting code in
https://go-review.googlesource.com/#/c/20097/ for go1.7.

"That is sort of surprising; the '0' and ' ' are applied per element when the slice is non-zero length"
The 2 is a bad test for the padding as %x will always print 2 places for strings/byteslices with a leading zero if need be.

The padding is always applied to the whole string because of:
"However, when printing a byte slice with a string-like verb (%s %q %x %X), it is treated identically to a string, as a single item."

https://play.golang.org/p/G5bwjqt4SP

%d is different from %x on strings and byte slices in that:

The %d on a slice is the recursive application:
"For compound operands such as slices and structs, the format applies to the elements of each operand, recursively, not to the operand as a whole."
So all the formatting applies to each element and fmt puts [ ] around the slice.

The %x is the special case:
"However, when printing a byte slice with a string-like verb (%s %q %x %X), it is treated identically to a string, as a single item."

And %x for a string has these rules:
' ': put spaces between bytes printing strings or slices in hex (% x, % X)
%x base 16, lower-case, two characters per byte
and the 0 modifies the padding which is only applied to the whole string.

So it seems consistent to me with the specific interpretation outlined as %d and %x are not both recursive applications of the integer format verbs %d and %x when applied to strings and byte slices.

@kortschak
Copy link
Contributor Author

@kortschak kortschak commented Sep 8, 2016

Thanks, Martin. TBH, I was just searching this morning for something to debug some byte slices, reading through the fmt_sbx, it's clear I wanted "%# x" as the format (which I will now remember). Maybe this is really just a documentation bug — a couple of examples perhaps?

@martisch
Copy link
Contributor

@martisch martisch commented Sep 8, 2016

examples: Sure if others agree that current behavior of padding the whole string should not be changed since it has been the case since at least 1.4?

Maybe 2 examples for byteslice formatting with %x under the section "However, when printing a byte slice with a string-like verb (%s %q %x %X), it is treated identically to a string, as a single item." ?

    fmt.Printf("%q\n", fmt.Sprintf("%#12x", b))  yields "    0x000102"
    fmt.Printf("%q\n", fmt.Sprintf("% #x", b))   yields "0x00 0x01 0x02"

or better adding another sentence for clarification what the effects are?

@robpike
Copy link
Contributor

@robpike robpike commented Sep 8, 2016

Perhaps it's best to document better rather than change the behavior. There are no executable examples in this package, and to provide them is a major undertaking. Maybe just a few words in the existing package documentation would help.

@rsc
Copy link
Contributor

@rsc rsc commented Oct 18, 2016

The fact that []byte is treated like string for the purposes of %s %q %x %X is documented and emphasized multiple times in the existing doc comment. I don't believe adding any new emphasis is going to help here.

As @martisch already mentioned:

For compound operands such as slices and structs, the format
applies to the elements of each operand, recursively, not to the
operand as a whole. Thus %q will quote each element of a slice
of strings, and %6.2f will control formatting for each element
of a floating-point array.

However, when printing a byte slice with a string-like verb
(%s %q %x %X), it is treated identically to a string, as a single item.
@rsc rsc closed this Oct 18, 2016
@golang golang locked and limited conversation to collaborators Oct 18, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.