time: time.FixedZone used most of memory when trasfer json marshaled time data among servers in different time zone

[locallocal]

What version of Go are you using (go version)?

$ go version
go version go1.15.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/test/.cache/go-build"
GOENV="/home/test/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/test/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/test/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io,direct"
GOROOT="/usr/lib/golang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build986830040=/tmp/go-build -gno-record-gcc-switches"

What did you do?

transfer struct data with json marshal betwen server and client, which includes three time fileds,
The special place is the server and cliet have differnt time zone.

What did you expect to see?

What did you see instead?

time.FixedZone uses most of the memory.

(pprof) top10
Showing nodes accounting for 10211.07MB, 99.46% of 10266.25MB total
Dropped 42 nodes (cum <= 51.33MB)
Showing top 10 nodes out of 33
flat flat% sum% cum cum%
4563.38MB 44.45% 44.45% 4563.38MB 44.45% time.FixedZone (inline)

[davecheney]

Thank you for raising this issue. Before we can proceed you will need to provide a small runnable go program that demonstrates the problem and someone else can use to reproduce the issue you are seeing.

Thank you

[gopherbot]

Change https://golang.org/cl/269817 mentions this issue: time: marshal and unmarshal time in UnixNano

[SpaceYang]

We have a filesystem service which written in go. there are some file meta information transferd between the fuse client and the meta server. we use json to marshal the struct. which include three time filed; beacue the time zone of client is different from the server. the client will create so many temporary objects named time.FixedZone when unmarshal the data. which leads to client OOM. so I think why we marshal time in uinx time(int64) and recreate the time with local time zone when unmarshal. PR #42577

[davecheney]

Thank you for the explanation. I see in the PR you sent that the issue is related to json. Can you please write a sample program to demonstrate the problem.

[SpaceYang]

Here is my code. but it can't run.

1. version1

Server: time zone is CST-8,

type InodeInfo struct {                                                                                                                                                                                                                   
    Inode      uint64     `json:"ino"`                                                        
    Mode       uint32     `json:"mode"`                                                        
    Nlink      uint32     `json:"nlink"`                                                        
    Size       uint64     `json:"sz"`                                                        
    Uid        uint32     `json:"uid"`                                                        
    Gid        uint32     `json:"gid"`                                                        
    Generation uint64     `json:"gen"`                                                        
    ModifyTime time.Time  `json:"mt"`                                                        
    CreateTime time.Time  `json:"ct"`                                                        
    AccessTime time.Time  `json:"at"`                                                        
    Target     []byte     `json:"tgt"`                                                        
    PInode     uint64     `json:"pino"`                                                        
} 

data := json.Marshal(inode)

after marshal the data,we transfer it to client with tcp and unmarshal the data.

Client: time zone is UTC0,

inodeInfo := new(InodeInfo)
err := json.Unmarshal(data, inodeInfo)

because the client and the server have different time zone, the client will create so many tempory timezone objects when unmarshal the data, and they take up a lot of memory,the details are as follows:

(pprof) top10
Showing nodes accounting for 10211.07MB, 99.46% of 10266.25MB total
Dropped 42 nodes (cum <= 51.33MB)
Showing top 10 nodes out of 33
flat flat% sum% cum cum%
4563.38MB 44.45% 44.45% 4563.38MB 44.45% time.FixedZone (inline)

2.version2(fix)

At last. we change our struct in order to solve the proble.

Server: time zone is CST-8,

type InodeInfo struct {                                                                                                                                                                                                                   
    Inode      uint64     `json:"ino"`                                                        
    Mode       uint32     `json:"mode"`                                                        
    Nlink      uint32     `json:"nlink"`                                                        
    Size       uint64     `json:"sz"`                                                        
    Uid        uint32     `json:"uid"`                                                        
    Gid        uint32     `json:"gid"`                                                        
    Generation uint64     `json:"gen"`  // with second                                                      
    ModifyTime int64      `json:"mt"`   // with second                                                     
    CreateTime int64      `json:"ct"`   // with second                                                  
    AccessTime int64      `json:"at"`                                                        
    Target     []byte     `json:"tgt"`                                                        
    PInode     uint64     `json:"pino"`                                                        
} 

Client: time zone is UTC0,

inodeInfo := new(InodeInfo)
err := json.Unmarshal(data, inodeInfo)
ModifyTime := time.Unix(inodeInfo.ModifyTime, 0)
CreateTime := time.Unix(inodeInfo.CreateTime, 0)
AccessTime := time.Unix(inodeInfo.AccessTime, 0)

Here is my idea, why we not marshal the time filed with UnixNano(int64) and use the local time zone to recate the time filed when we unmarshal the struct in standard package time. so I create this issue and a PR.

[networkimprov]

cc @mvdan

@gopherbot remove WaitingForInfo

[mvdan]

I'm not sure I understand this issue. If you don't like that time.Time marshals and unmarshals to a format with a timezone, why don't you just marshal and unmarshal its UnixNano integer directly?

The change you proposed would break existing programs, so we definitely cannot do that.

There's always the option of trying to optimize the existing code without changing the API. For example, I see that the code does Parse(`"`+RFC3339+`"`, string(data)), which seems a bit wasteful - we probably make a copy of both the format and the incoming data, neither of which is truly necessary.

[SpaceYang]

@mvdan,Thank you for your reply.Just like what you say,we marshal its UnixNano directly after analysis.but at the begging, we don't konw the temporary variable time.FixedZone would take up so much memory when we umarshal the data , which created by Parse function you mentioned.Maybe there are other better ways to optimize.I will close this issue.

[cagedmantis]

I've closed this issue since @SpaceYang mentioned they would close it. Please reopen this issue if you believe the issue was closed in error.