[filechunk_manifest.go:190] read http://xxxx failed, err: unexpected EOF

[kmlebedev]

Describe the bug
Bulk read chunk error

[filechunk_manifest.go:190] read http://fast-volume-3:8080/957,2a7a3800944f67d09f322d81 failed, err: unexpected EOF

volume server 3.09

2022/06/14 11:35:27 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:329)

System Setup

3.10

[kmlebedev]

@chrislusf The problem seems to be here 4fd5f96 or ebd6097 or ecef844

[kmlebedev]

add logging error

E0614 17:22:36     1 common.go:329] processRangeRequest ReadNeedleData write: http: wrote more than the declared Content-Length
2022/06/14 17:22:36 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:330)

[kmlebedev]

@chrislusf Please help, i found error line is "ReadNeedleData write" https://github.com/chrislusf/seaweedfs/blob/master/weed/storage/needle/needle_read_page.go#L20

func (n *Needle) ReadNeedleDataInto(r backend.BackendStorageFile, volumeOffset int64, buf []byte, writer io.Writer, needleOffset int64, size int64) (err error) {
	crc := CRC(0)
	for x := needleOffset; x < needleOffset+size; x += int64(len(buf)) {
		count, err := n.ReadNeedleData(r, volumeOffset, buf, x)
		if count > 0 {
			crc = crc.Update(buf[0:count])
			if _, err = writer.Write(buf[0:count]); err != nil {
				return fmt.Errorf("ReadNeedleData write: %v", err)
			}
		}

[kmlebedev]

Add logging headers

E0614 17:42:17     1 common.go:329] processRangeRequest headers: map[Accept-Ranges:[bytes] Content-Disposition:[inline; filename="0001.part"] Content-Length:[5] Content-Range:[bytes 0-4/31457280] Etag:["77371355"] Last-Modified:[Tue, 14 Jun 2022 15:00:44 GMT] Server:[SeaweedFS Volume 30GB 3.10]] err: ReadNeedleData write: http: wrote more than the declared Content-Length

All files have Content-Length:[5] for Content-Range:[bytes 0-4/XXXXXX]

and

E0614 17:42:05     1 common.go:329] processRangeRequest headers: map[Accept-Ranges:[bytes] Content-Disposition:[inline; filename="0004.part"] Content-Length:[143456] Content-Range:[bytes 11558288-11701743/16777216] Etag:["fddf02c7"] Last-Modified:[Tue, 14 Jun 2022 07:48:57 GMT] Server:[SeaweedFS Volume 30GB 3.10]] err: ReadNeedleData write: http: wrote more than the declared Content-Length

[kmlebedev]

ReadNeedleData logging

E0614 18:12:24     1 common.go:329] processRangeRequest headers: map[Accept-Ranges:[bytes] Content-Disposition:[inline; filename="0019.part"] Content-Length:[4896] Content-Range:[bytes 7236048-7240943/16777216] Etag:["70b23b81"] Last-Modified:[Tue, 14 Jun 2022 05:46:21 GMT] Server:[SeaweedFS Volume 30GB 3.10]] err: ReadNeedleData len(buf): 1048576 x: 7236048 count: 1048576 volumeOffset: 252041008 needleOffset: 7236048 size: 4896 write: http: wrote more than the declared Content-Length
E0614 18:13:06     1 common.go:329] processRangeRequest headers: map[Accept-Ranges:[bytes] Content-Disposition:[inline; filename="0019.part"] Content-Length:[12] Content-Range:[bytes 7255240-7255251/16777216] Etag:["70b23b81"] Last-Modified:[Tue, 14 Jun 2022 05:46:21 GMT] Server:[SeaweedFS Volume 30GB 3.10]] err: ReadNeedleData len(buf): 1048576 x: 7255240 count: 1048576 volumeOffset: 252041008 needleOffset: 7255240 size: 12 write: http: wrote more than the declared Content-Length

[kmlebedev]

@chrislusf I confirm the fix helped