Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seems the FreelistMapType has a poor delete performance while nosync #239

Open
absolute8511 opened this issue Aug 27, 2020 · 0 comments
Open

Comments

@absolute8511
Copy link

absolute8511 commented Aug 27, 2020

I wrote a simple benchmark to reproduce the compare result.

func BenchmarkBoltDBDeleteUsingMap(b *testing.B) {
	benchmarkBoltDBDelete(b, bolt.FreelistMapType)
}
func BenchmarkBoltDBDeleteUsingArray(b *testing.B) {
	benchmarkBoltDBDelete(b, bolt.FreelistArrayType)
}
func benchmarkBoltDBDelete(b *testing.B, listType bolt.FreelistType) {
	o := &bolt.Options{
		FreelistType: listType,
		NoSync:       true,
	}
	f := tempfile()
	bdb, err := bolt.Open(f, 0666, o)
	if err != nil {
		panic(err)
	}
	db := &DB{
		DB: bdb,
		f:  f,
		o:  o,
	}
	defer db.MustClose()
	if err := db.Update(func(tx *bolt.Tx) error {
		_, err := tx.CreateBucket([]byte("bench"))
		return err
	}); err != nil {
		b.Fatal(err)
	}

	buf := make([]byte, 4)
	for i := 0; i < b.N; i++ {
		binary.LittleEndian.PutUint32(buf, uint32(i))
		insert := func(tx *bolt.Tx) error {
			b := tx.Bucket([]byte("bench"))
			b.Put(buf, []byte("filler"))
			return nil
		}
		if err := db.Update(insert); err != nil {
			b.Error(err)
			return
		}
	}

	b.ReportAllocs()
	b.ResetTimer()
	s := time.Now()
	buf = make([]byte, 4)
	for i := 0; i < b.N; i++ {
		binary.LittleEndian.PutUint32(buf, uint32(i))
		delop := func(tx *bolt.Tx) error {
			bu := tx.Bucket([]byte("bench"))
			v := bu.Get(buf)
			if v == nil {
				b.Errorf("value empty")
			}
			bu.Delete(buf)
			return nil
		}
		if err := db.Update(delop); err != nil {
			b.Error(err)
			return
		}
	}

	cost := time.Since(s)
	b.Logf("cost %s", cost)
	b.StopTimer()
}

the benchmark showed as below:

$ go test -run xxx -bench BenchmarkBoltDBDelete -benchtime=10s
seed: 62743
quick settings: count=5, items=1000, ksize=1024, vsize=1024
goos: linux
goarch: amd64
pkg: github.com/absolute8511/bolt
BenchmarkBoltDBDeleteUsingMap-4     	  257385	    171098 ns/op	   35214 B/op	      77 allocs/op
--- BENCH: BenchmarkBoltDBDeleteUsingMap-4
    db_test.go:1904: cost 27.924µs
    db_test.go:1904: cost 2.909692ms
    db_test.go:1904: cost 466.201181ms
    db_test.go:1904: cost 44.037952974s
BenchmarkBoltDBDeleteUsingArray-4   	  265882	     58489 ns/op	   35972 B/op	      65 allocs/op
--- BENCH: BenchmarkBoltDBDeleteUsingArray-4
    db_test.go:1904: cost 25.178µs
    db_test.go:1904: cost 2.068921ms
    db_test.go:1904: cost 451.299584ms
    db_test.go:1904: cost 15.551035833s
PASS
ok  	github.com/absolute8511/bolt	95.002s

using FreelistMapType the performance is worse compared to the FreelistArrayType.(about 30% more) I am wondering is it expected?

By profile on the test, I found the getFreePageIDs for MapType is much slower than the ArrayType.

@absolute8511 absolute8511 changed the title Seems the FreelistMapType has a poor delete performance Seems the FreelistMapType has a poor delete performance on large data Aug 27, 2020
@absolute8511 absolute8511 changed the title Seems the FreelistMapType has a poor delete performance on large data Seems the FreelistMapType has a poor delete performance while nosync Aug 28, 2020
@absolute8511 absolute8511 reopened this Aug 28, 2020
@github-actions github-actions bot added the stale label May 10, 2024
@ahrtr ahrtr removed the stale label May 10, 2024
@github-actions github-actions bot added the stale label Aug 14, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 4, 2024
@ahrtr ahrtr reopened this Sep 4, 2024
@github-actions github-actions bot removed the stale label Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants