archive/zip: does not allow local filename encoding

[k-hiro]

I making zip file with local filename encoding for compatibility on japanese Windows.
Below codes was working fine with go 1.8.4 windows/amd64.

package main

import (
  "archive/zip"
  "fmt"
  "os"

  "golang.org/x/text/encoding/japanese"
  "golang.org/x/text/transform"
)

func main() {
  f, err := os.Create("test.zip")
  if err != nil {
    panic(err)
  }

  myZip := zip.NewWriter(f)
  defer myZip.Close()

  jpnName, _, err := transform.String(japanese.ShiftJIS.NewEncoder(), "日本語.txt")
  if err != nil {
    panic(err)
  }

  header := zip.FileHeader{
    Name:   jpnName,
    Method: zip.Deflate,
  }

  writer, err := myZip.CreateHeader(&header)
  if err != nil {
    panic(err)
  }

  fmt.Fprintln(writer, "hello")
}

With go 1.9.1 windows/amd64, CreateHeader() overwrites utf8 flag to 1. So there is no way to create zip with local encoding.

Even today, Windows 7 can't extract utf-8 encoded zip by default.
User needs to manually download and install HOTFIX to extract correctly.

Windows 8 and later can extract utf-8 encoded zip without problems.
But Windows zip tool doesn't make any utf-8 encoded zip file. it always uses local encoding.
If you try to compress files that includes unicode only characters like emoji in filename, error dialog popups and it says like "you can't use this filename, please change it."
Also 7zip for Windows uses local encoding by default. you can make utf-8 encoded zip with -mcu=on option.

I think current implementation is intended to set utf8 flag only if filename is valid utf-8 string. But hasValidUTF8() function returns true for non ascii string even if its not valid utf-8.

go/src/archive/zip/writer.go

Lines 218 to 230 in 2da8a16

	func hasValidUTF8(s string) bool {
	n := 0
	for _, r := range s {
	// By default, ZIP uses CP437, which is only identical to ASCII for the printable characters.
	if r < 0x20 || r >= 0x7f {
	if !utf8.ValidRune(r) {
	return false
	}
	n++
	}
	}
	return n > 0
	}

see also #10741

[ianlancetaylor]

CC @mattn

[dsnet]

Closing as duplicate of #10741. I have finished all planned improvements to archive/tar. I'll look to improving and cleaning up archive/zip. Probably won't be able to get much done in the remainder of this cycle, though.

Please continue discussion on #10741

[gopherbot]

Change https://golang.org/cl/72791 mentions this issue: archive/zip: restrict UTF8 detection for comment and name fields