[volume_info] load volume 16.vif file: unmarshal error: unexpected EOF #2633

kmlebedev · 2022-02-04T08:31:03Z

weed version

2.87

I got a lot of errors after restarting volumes

logs

  | Jan 29, 2022 @ 15:09:18.884 | W0129 10:09:17     1 volume_tier.go:31] load volume 286.vif file: unmarshal error: unexpected EOF | fast-volume-0
  | Jan 29, 2022 @ 15:09:18.884 | W0129 10:09:17     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-0
  | Jan 29, 2022 @ 20:03:42.946 | W0129 15:03:36     1 volume_tier.go:31] load volume 94.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.946 | W0129 15:03:36     1 volume_tier.go:31] load volume 255.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.946 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.946 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.946 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.946 | W0129 15:03:36     1 volume_tier.go:31] load volume 253.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_tier.go:31] load volume 173.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_tier.go:31] load volume 172.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_tier.go:31] load volume 227.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_tier.go:31] load volume 263.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.947 | W0129 15:03:36     1 volume_tier.go:31] load volume 185.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.948 | W0129 15:03:36     1 volume_tier.go:31] load volume 52.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.948 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.948 | W0129 15:03:36     1 volume_tier.go:31] load volume 248.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.948 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_tier.go:31] load volume 223.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_tier.go:31] load volume 237.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_tier.go:31] load volume 35.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_tier.go:31] load volume 36.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_tier.go:31] load volume 40.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.949 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_tier.go:31] load volume 97.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_tier.go:31] load volume 18.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_tier.go:31] load volume 71.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_tier.go:31] load volume 16.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_tier.go:31] load volume 133.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.950 | W0129 15:03:36     1 volume_tier.go:31] load volume 178.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_tier.go:31] load volume 109.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_tier.go:31] load volume 199.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_tier.go:31] load volume 200.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.951 | W0129 15:03:36     1 volume_tier.go:31] load volume 267.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_tier.go:31] load volume 141.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_tier.go:31] load volume 13.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_tier.go:31] load volume 14.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.953 | W0129 15:03:36     1 volume_tier.go:31] load volume 48.vif file: unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.954 | W0129 15:03:36     1 volume_info.go:48] unmarshal error: unexpected EOF | fast-volume-1
  | Jan 29, 2022 @ 20:03:42.954 | W0129 15:03:36     1 volume_tier.go:31] load volume 163.vif file: unmarshal error: unexpected EOF | fast-volume-1

for f in $(ls *.vif); do fc=$(cat $f); if [[ "${fc: -1}" != "}" ]];then ls -lu $f;cat $f; echo;fi; done
-rw-r--r--    1 root     99               8 Feb  4 07:50 254.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 95.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 172.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 185.vif
{
  "fil
-rw-rw-r--    1 root     99               8 Feb  4 07:50 65.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 52.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 50.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 232.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 190.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 40.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 36.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 237.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 223.vif
{
  "fil
-rw-rw-r--    1 root     99               8 Feb  4 07:50 16.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 133.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 199.vif
{
  "fil
-rw-rw-r--    1 root     99               8 Feb  4 07:49 259.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 109.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 81.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 141.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 44.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 13.vif
{
  "fil
-rw-r--r--    1 root     99               8 Feb  4 07:50 274.vif
{
  "fil
-rw-rw-r--    1 root     99               8 Feb  4 07:49 163.vif
{
  "fil

It seems that somewhere there is not enough fsync for vif files

The text was updated successfully, but these errors were encountered:

chrislusf · 2022-02-04T09:09:28Z

what is the 8 bytes in the .vif files?

kmlebedev · 2022-02-04T09:12:11Z

what is the 8 bytes in the .vif files?

hexdump  -v 163.vif
0000000 0a7b 2020 6622 6c69
0000008

cat 163.vif

{
  "fil

chrislusf · 2022-02-04T09:19:22Z

https://github.com/chrislusf/seaweedfs/blob/master/weed/storage/volume_info/volume_info.go#L78
is the only place writing this file. But I do not know how to improve here.

chrislusf · 2022-02-04T09:19:56Z

Why it is closed?

kmlebedev · 2022-02-04T09:21:07Z

Why it is closed?

accident

kmlebedev · 2022-02-04T11:16:51Z

https://github.com/chrislusf/seaweedfs/blob/master/weed/storage/volume_info/volume_info.go#L78 is the only place writing this file. But I do not know how to improve here.

golang/go#20599

https://github.com/tuenti/pouch/pull/20/files

chrislusf · 2022-02-04T18:02:59Z

Good to learn this! Thanks for the finding!

fix seaweedfs#2633

kmlebedev · 2022-04-08T06:32:01Z

Repeated on version 2.90

/data # ls -l /data/main_99.vif
-rw-rw-r--    1 root     99               8 Feb 10 10:44 /data/main_99.vif
/data # ls -l /data/current_93.vif
-rw-rw-r--    1 root     99               8 Feb 10 12:09 /data/current_93.vif
/data # echo volume.list | weed shell | grep "id:99"
          volume id:99  size:8  collection:"main"  replica_placement:100  version:3  compact_revision:5  modified_at_second:1646345554
          volume id:99  size:8  collection:"main"  replica_placement:100  version:3  compact_revision:5  modified_at_second:1646345554
/data # echo volume.list | weed shell | grep "id:93"
          volume id:93  size:8  collection:"current"  replica_placement:100  version:3  compact_revision:18  modified_at_second:1648840084
          volume id:93  size:8  collection:"current"  replica_placement:100  version:3  compact_revision:18  modified
_at_second:1648840084

hexdump -vC /data/main_99.vif
00000000  7b 0a 20 20 22 66 69 6c                           |{.  "fil|
00000008

adamlamar · 2022-07-11T18:16:29Z

Does this bug lead to "data loss"? I have a number of volumes with 8 byte .vif files and some objects on my cluster are inaccessible. Can I just replace the 8 byte version with the 57 byte version that all the working .vif files have?

chrislusf · 2022-07-11T18:25:52Z

Can I just replace the 8 byte version with the 57 byte version that all the working .vif files have?

Yes, or you can simply the .vif file in this case.

adamlamar · 2022-07-11T18:28:48Z

Roger, I will find all the 8 byte .vif files and delete them, then restart my volume servers. I confirmed that I was able to access the objects again after replacing the .vif with a new one and restarting the volume server. Thanks Chris!

kmlebedev · 2023-10-20T11:02:20Z

#3528

kmlebedev · 2023-10-24T06:42:24Z

@chrislusf Hi, let's reopen this issue.
#4934 (comment)

closes seaweedfs#4934 fixes seaweedfs#2633 might fix seaweedfs#3528

closes #4934 fixes #2633 might fix #3528

closes seaweedfs#4934 fixes seaweedfs#2633 might fix seaweedfs#3528

kmlebedev closed this as completed Feb 4, 2022

kmlebedev reopened this Feb 4, 2022

chrislusf closed this as completed in 76e297d Feb 4, 2022

Radtoo pushed a commit to Radtoo/seaweedfs that referenced this issue Feb 6, 2022

sync call to write file, avoid vif loading error

ce96982

fix seaweedfs#2633

zehweh added a commit to zehweh/seaweedfs that referenced this issue Oct 24, 2023

fix copying .vif file in VolumeCopy

ef55031

closes seaweedfs#4934 fixes seaweedfs#2633 might fix seaweedfs#3528

This was referenced Oct 24, 2023

fix copying .vif file in VolumeCopy zehweh/seaweedfs#1

Merged

fix copying .vif file in VolumeCopy #4943

Merged

chrislusf pushed a commit that referenced this issue Oct 24, 2023

fix copying .vif file in VolumeCopy (#4943)

8b39bbb

closes #4934 fixes #2633 might fix #3528

kmlebedev pushed a commit to kmlebedev/seaweedfs that referenced this issue Dec 22, 2023

fix copying .vif file in VolumeCopy (seaweedfs#4943)

9f8e1f4

closes seaweedfs#4934 fixes seaweedfs#2633 might fix seaweedfs#3528

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[volume_info] load volume 16.vif file: unmarshal error: unexpected EOF #2633

[volume_info] load volume 16.vif file: unmarshal error: unexpected EOF #2633

kmlebedev commented Feb 4, 2022 •

edited

chrislusf commented Feb 4, 2022

kmlebedev commented Feb 4, 2022

chrislusf commented Feb 4, 2022

chrislusf commented Feb 4, 2022

kmlebedev commented Feb 4, 2022

kmlebedev commented Feb 4, 2022 •

edited

chrislusf commented Feb 4, 2022

kmlebedev commented Apr 8, 2022 •

edited

adamlamar commented Jul 11, 2022

chrislusf commented Jul 11, 2022

adamlamar commented Jul 11, 2022 •

edited

kmlebedev commented Oct 20, 2023

kmlebedev commented Oct 24, 2023 •

edited

[volume_info] load volume 16.vif file: unmarshal error: unexpected EOF #2633

[volume_info] load volume 16.vif file: unmarshal error: unexpected EOF #2633

Comments

kmlebedev commented Feb 4, 2022 • edited

chrislusf commented Feb 4, 2022

kmlebedev commented Feb 4, 2022

chrislusf commented Feb 4, 2022

chrislusf commented Feb 4, 2022

kmlebedev commented Feb 4, 2022

kmlebedev commented Feb 4, 2022 • edited

chrislusf commented Feb 4, 2022

kmlebedev commented Apr 8, 2022 • edited

adamlamar commented Jul 11, 2022

chrislusf commented Jul 11, 2022

adamlamar commented Jul 11, 2022 • edited

kmlebedev commented Oct 20, 2023

kmlebedev commented Oct 24, 2023 • edited

kmlebedev commented Feb 4, 2022 •

edited

kmlebedev commented Feb 4, 2022 •

edited

kmlebedev commented Apr 8, 2022 •

edited

adamlamar commented Jul 11, 2022 •

edited

kmlebedev commented Oct 24, 2023 •

edited