Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve robustness to broken input files #12

Merged
merged 3 commits into from
Jan 10, 2019
Merged

Conversation

satta
Copy link
Member

@satta satta commented Jan 10, 2019

One can craft a Bloom filter file in which the number of hash functions (k) is an unsigned integer larger than the maximum signed integer on the given platform. This will cause a crash later when trying to make() a slice with that overly large size, as the length of a slice in Go is limited by the maximum value of (signed) int:

$ echo "F" | bloom check testdata/broken.bloom
panic: runtime error: makeslice: len out of range

goroutine 1 [running]:
github.com/DCSO/bloom.(*BloomFilter).Check(0xc4200741e0, 0xc420016428, 0x1, 0x8, 0x1)
	/home/satta/golang/src/github.com/DCSO/bloom/bloom.go:275 +0x44
main.checkAgainstFilter(0xc4200c4050, 0x42, 0x0, 0x5f66d9, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/satta/golang/src/github.com/DCSO/bloom/bloom/manage.go:169 +0x1ce
main.main.func4(0xc4200ce420, 0x0, 0xc4200ce420)
	/home/satta/golang/src/github.com/DCSO/bloom/bloom/manage.go:412 +0x11f
gopkg.in/urfave/cli%2ev1.HandleAction(0x5b3ea0, 0x602468, 0xc4200ce420, 0xc420074100, 0x0)
	/home/satta/golang/src/gopkg.in/urfave/cli.v1/app.go:490 +0xc8
gopkg.in/urfave/cli%2ev1.Command.Run(0x5f6c21, 0x5, 0x0, 0x0, 0xc420058510, 0x1, 0x1, 0x60009e, 0x2f, 0x0, ...)
	/home/satta/golang/src/gopkg.in/urfave/cli.v1/command.go:210 +0xa36
gopkg.in/urfave/cli%2ev1.(*App).Run(0xc4200a8340, 0xc420010180, 0x3, 0x3, 0x0, 0x0)
	/home/satta/golang/src/gopkg.in/urfave/cli.v1/app.go:255 +0x6a0
main.main()
	/home/satta/golang/src/github.com/DCSO/bloom/bloom/manage.go:476 +0xd8c

It should be unlikely to create such a filter using the regular initializer, but such a filter can be read from a corrupted file.

This PR makes sure we fail early at load time when we notice that this value exceeds the given limit.

One can craft a Bloom filter file in which the number of hash functions
(k) is an unsigned integer larger than the maximum signed integer on the
given platform. This will cause a crash later when trying to make() a
slice with that overly large size, as the length of a slice in Go is
limited by the maximum value of (signed) int.
It should be unlikely to create such a filter using the regular
initializer, but such as filter can be read from a corrupted file.
Here, we fail early at load time when we notice that this value exceeds
the given limit.
io_test.go Show resolved Hide resolved
@rhaist rhaist merged commit 691ea61 into DCSO:master Jan 10, 2019
@satta satta deleted the broken-crash branch January 10, 2019 14:50
@satta satta assigned satta and unassigned satta Jan 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants