encoding/base64: NUL byte in input leads to wrong output

[gopherbot]

by mstplbrg:

What steps will reproduce the problem?
1. Run this go program: http://play.golang.org/p/J6885vjGRj — it base64-encodes
"foo\x00bar"
2. Run base64 -d on the output ("echo Zm9vAGJh | base64 -d")
3. You get "fooba" instead of "foo\0bar"

What is the expected output?
Zm9vAGJhcgo=

What do you see instead?
Zm9vAGJh

Which compiler are you using (5g, 6g, 8g, gccgo)?
8g

Which operating system are you using?
Linux (Debian testing)

Which version are you using?  (run 'go version')
go version devel +4d46cae98846 Thu Nov 15 11:46:00 2012 -0800

[robpike]

Comment 1:

I believe the code is working and that your test is confused by the NUL. If I s/d/D/ (on
mac) in your test, I see your output, but this code works as it should:
http://play.golang.org/p/OzNsC8fhDB
I believe the base64 tool or Unix output routines may be at fault here, not the Go
library.
Now, the string you expect to see, ending in =, produces a different result when I
decode it: it includes a newline.
http://play.golang.org/p/JSFuw7sNEo
I believe things are working as intended but it's confusing enough I'll leave it open
and let you respond. Case is not yet proven to my satisfaction.

Status changed to WaitingForReply.

[gopherbot]

Comment 2 by mstplbrg:

Thank you for your reply.
You are indeed correct that my expected output from my original report contains a
newline. Here is how to fix it:
$ echo -n 'foo\0bar' | base64
Zm9vAGJhcg==
This value (Zm9vAGJhcg==) is still different from the Go result (Zm9vAGJh).
The code you posted as "works as it should" (http://play.golang.org/p/OzNsC8fhDB)
produces the following output when run on play.golang.org:
"Zm9vAGJh"
"foo\x00ba"
The problem here is that the last "r" is missing. That is, the output should be
"foo\x00bar", not "foo\x00ba".
I hope this clarifies things :-). Thanks for looking into this!

[robpike]

Comment 3:

Labels changed: added priority-soon, removed priority-triage.

Status changed to Accepted.

[gopherbot]

Comment 4 by mstplbrg:

I figured out what the problem was:
I neglected to call encoder.Close(), even though the documentation states it is
necessary. So turns out the \0 is not the problem and in fact works as it should :-).
This issue can be closed.

[minux]

Comment 5:

I copied your test case and turned it into an example for encoding/base64
to emphasize the importance of encoder.Close().
http://golang.org/cl/6847125

[minux]

Comment 6:

This issue was closed by revision 15353d2.

Status changed to Fixed.