Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash on decoding value using CLI #1399

Closed
ammmir opened this issue Jul 8, 2020 · 5 comments
Closed

Crash on decoding value using CLI #1399

ammmir opened this issue Jul 8, 2020 · 5 comments
Labels
status/needs-specs Issues that require further specification before implementation.

Comments

@ammmir
Copy link
Contributor

ammmir commented Jul 8, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.3 darwin/amd64

What operating system are you using?

MacOS X

What version of Badger are you using?

2.0.3

Does this issue reproduce with the latest master?

Not tested

What Badger options were set?

default

What did you do?

Database was migrated from manifest v4 to v7 (badger 1.6.0 or thereabouts to 2.0.3) via badger backup/restore.

badger info --dir db --show-keys --with-prefix 019a4730021f457ba087a255a58f49 --read-only=false

What did you see instead?

╰─➤  /tmp/badger2 info --dir db --show-keys --with-prefix 019a4730021f457ba087a255a58f49 --read-only=false
Listening for /debug HTTP requests at port: 8080

[2020-07-08T15:58:40+09:00] MANIFEST       50 B MA
[        6 minutes earlier] 000008.sst    15 MB L1
[       40 minutes earlier] 000000.vlog   96 MB VL
[       40 minutes earlier] 000002.vlog  1.3 GB VL
[                      now] 000003.vlog   32 MB VL

[EXTRA]
[2020-07-08T15:18:08+09:00] KEYREGISTRY    28 B

[Summary]
Level 0 size:          0 B
Level 1 size:        15 MB
Total index size:    15 MB
Value log size:     1.4 GB

Abnormalities:
1 extra file.
0 missing files.
0 empty files.
0 truncated manifests.
badger 2020/07/08 15:58:42 INFO: All 1 tables opened in 10ms
badger 2020/07/08 15:58:42 INFO: Replaying file id: 3 at offset: 32266481
badger 2020/07/08 15:58:42 INFO: Replay took: 5.337µs
badger 2020/07/08 15:58:42 DEBUG: Value log discard stats empty
Only choosing keys with prefix:
00000000  01 9a 47 30 02 1f 45 7b  a0 87 a2 55 a5 8f 49     |..G0..E{...U..I|
badger 2020/07/08 15:58:42 INFO: Got compaction priority: {level:0 score:1.73 dropPrefix:[]}
panic: runtime error: slice bounds out of range [:12] with capacity 0

goroutine 1 [running]:
github.com/dgraph-io/badger/v2.(*valuePointer).Decode(...)
	/Users/amir/src/badger/structs.go:61
github.com/dgraph-io/badger/v2.(*Item).EstimatedSize(...)
	/Users/amir/src/badger/iterator.go:247
github.com/dgraph-io/badger/v2/badger/cmd.printKey(0xc00019c6e0, 0xc0001b0000, 0x10, 0x10)
	/Users/amir/src/badger/badger/cmd/info.go:211 +0x61c
github.com/dgraph-io/badger/v2/badger/cmd.showKeys(0xc000330000, 0xc0000c8100, 0xf, 0x20, 0x0, 0x0)
	/Users/amir/src/badger/badger/cmd/info.go:151 +0x257
github.com/dgraph-io/badger/v2/badger/cmd.handleInfo(0x4aa0840, 0xc0001888a0, 0x0, 0x6, 0x0, 0x0)
	/Users/amir/src/badger/badger/cmd/info.go:120 +0x4ce
github.com/spf13/cobra.(*Command).execute(0x4aa0840, 0xc000188840, 0x6, 0x6, 0x4aa0840, 0xc000188840)
	/Users/amir/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:826 +0x453
github.com/spf13/cobra.(*Command).ExecuteC(0x4aa0fc0, 0x403aee0, 0x4a5c728, 0x0)
	/Users/amir/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914 +0x2fb
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/amir/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/badger/v2/badger/cmd.Execute()
	/Users/amir/src/badger/badger/cmd/root.go:40 +0x31
main.main()
	/Users/amir/src/badger/badger/main.go:41 +0x54

If --with-prefix is omitted it doesn't crash.
@jarifibrahim
Copy link
Contributor

There was a bug in backup restore code which could've caused this 9459a24 .

@ammmir would it be possible for you to do a backup restore from your v1.6x badger to the master version?

@jarifibrahim jarifibrahim added the status/needs-specs Issues that require further specification before implementation. label Jul 8, 2020
@ammmir
Copy link
Contributor Author

ammmir commented Jul 9, 2020

Thanks for the quick reply @jarifibrahim -- I've pulled master and tried to restore the backup, but with the same result:


Listening for /debug HTTP requests at port: 8080

[2020-07-09T12:23:04+09:00] MANIFEST       50 B MA
[         20 hours earlier] 000008.sst    15 MB L1
[         21 hours earlier] 000000.vlog   96 MB VL
[         21 hours earlier] 000002.vlog  1.3 GB VL
[                      now] 000003.vlog   32 MB VL

[EXTRA]
[2020-07-08T15:18:08+09:00] KEYREGISTRY    28 B

[Summary]
Level 0 size:          0 B
Level 1 size:        15 MB
Total index size:    15 MB
Value log size:     1.4 GB

Abnormalities:
1 extra file.
0 missing files.
0 empty files.
0 truncated manifests.
badger 2020/07/09 12:24:03 INFO: All 1 tables opened in 9ms
badger 2020/07/09 12:24:03 INFO: Replaying file id: 3 at offset: 32266481
badger 2020/07/09 12:24:03 INFO: Replay took: 7.614µs
Only choosing keys with prefix:
00000000  01 9a 47 30 02 1f 45 7b  a0 87 a2 55 a5 8f 49     |..G0..E{...U..I|
badger 2020/07/09 12:24:03 INFO: Got compaction priority: {level:0 score:1.73 dropPrefixes:[]}
panic: runtime error: slice bounds out of range [:12] with capacity 0

goroutine 1 [running]:
github.com/dgraph-io/badger/v2.(*valuePointer).Decode(...)
	/Users/amir/src/badger/structs.go:61
github.com/dgraph-io/badger/v2.(*Item).EstimatedSize(...)
	/Users/amir/src/badger/iterator.go:251
github.com/dgraph-io/badger/v2/badger/cmd.printKey(0xc005720210, 0xc005702200, 0x10, 0x10)
	/Users/amir/src/badger/badger/cmd/info.go:211 +0x61c
github.com/dgraph-io/badger/v2/badger/cmd.showKeys(0xc0001ca400, 0xc000097160, 0xf, 0x20, 0x0, 0x0)
	/Users/amir/src/badger/badger/cmd/info.go:151 +0x257
github.com/dgraph-io/badger/v2/badger/cmd.handleInfo(0x4aa9860, 0xc000180960, 0x0, 0x6, 0x0, 0x0)
	/Users/amir/src/badger/badger/cmd/info.go:120 +0x44d
github.com/spf13/cobra.(*Command).execute(0x4aa9860, 0xc000180900, 0x6, 0x6, 0x4aa9860, 0xc000180900)
	/Users/amir/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:826 +0x453
github.com/spf13/cobra.(*Command).ExecuteC(0x4aa9fe0, 0x403b070, 0x4a65728, 0x0)
	/Users/amir/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914 +0x2fb
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/amir/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/badger/v2/badger/cmd.Execute()
	/Users/amir/src/badger/badger/cmd/root.go:40 +0x31
main.main()
	/Users/amir/src/badger/badger/main.go:41 +0x54

Should I regenerate the backup?

@jarifibrahim
Copy link
Contributor

Can you make a copy of your data directory and share it with me? I'd like to reproduce the bug on my end.

@ammmir Yes, try doing a new backup and restore but please keep the old directory.

@ammmir
Copy link
Contributor Author

ammmir commented Jul 19, 2020

@jarifibrahim unfortunately, I can't send any files since it's customer data. is there anything else that might help you?

if nothing else, I'll write my own backup tool to export the data from v1 and reimport to v2, since all I really care about are the keys and values, TTL, and UserMeta.

@minhaj-shakeel
Copy link

Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.

drawing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status/needs-specs Issues that require further specification before implementation.
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ammmir @jarifibrahim @minhaj-shakeel