Skip to content
This repository has been archived by the owner on Mar 9, 2019. It is now read-only.

page ... already freed #574

Open
k0mrade opened this issue Jul 18, 2016 · 4 comments
Open

page ... already freed #574

k0mrade opened this issue Jul 18, 2016 · 4 comments

Comments

@k0mrade
Copy link

k0mrade commented Jul 18, 2016 •
edited
Loading

Hi, catched the error

server.go:2161: http: panic serving 127.0.0.1:41609: page 152426 already freed
Jul 18 08:26:30 vh3 aveplaza[12682]: goroutine 83890 [running]:
Jul 18 08:26:30 vh3 aveplaza[12682]: net/http.(*conn).serve.func1(0xc8205f7300)
Jul 18 08:26:30 vh3 aveplaza[12682]: /usr/local/go/src/net/http/server.go:1389 +0xc1
Jul 18 08:26:30 vh3 aveplaza[12682]: panic(0x8df1e0, 0xc82012b860)
Jul 18 08:26:30 vh3 aveplaza[12682]: /usr/local/go/src/runtime/panic.go:443 +0x4e9
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*freelist).free(0xc820150cc0, 0xeab07, 0x7f5b1936a000)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/freelist.go:117 +0x37b
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*node).spill(0xc820213e30, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/node.go:358 +0x2ca
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*node).spill(0xc820213dc0, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/node.go:345 +0x12c
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*Bucket).spill(0xc820597a80, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/bucket.go:541 +0x1cf
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*Bucket).spill(0xc820597a40, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/bucket.go:508 +0xbc2
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*Bucket).spill(0xc8202602b8, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/bucket.go:508 +0xbc2
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*Tx).Commit(0xc8202602a0, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/tx.go:163 +0x1d4
Jul 18 08:26:30 vh3 aveplaza[12682]: github.com/boltdb/bolt.(*DB).Update(0xc8201601e0, 0xc82050efe8, 0x0, 0x0)
Jul 18 08:26:30 vh3 aveplaza[12682]: /go/src/github.com/boltdb/bolt/db.go:602 +0x11d
Jul 18 08:26:30 vh3 aveplaza[12682]: aveplaza/shared/advertisement.SessionSave(0xa925c0, 0x8, 0xc8201bb54c, 0x11, 0xc8201bb4c6, 0x33, 0xc8201bb566, 0x23, 0x14624ba2a1e00224, 0xa1b203eb3d1a0000, ...)

Check returns
bolt check db.back.db > 1.txt

page 152462: reachable freed
page 26514: multiple references
page 26514: multiple references
......
......
......
page 152432: multiple references
page 37332: multiple references
page 152451: multiple references
page 152451: invalid type: freelist

panic: invalid page type: 152451: 10

goroutine 3 [running]:
github.com/boltdb/bolt.(*Cursor).search(0xc820367490, 0x25840030, 0x9b, 0x7fffffff, 0x25383)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cursor.go:256 +0x215
github.com/boltdb/bolt.(*Cursor).searchPage(0xc820367490, 0x25840030, 0x9b, 0x7fffffff, 0x25860000)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cursor.go:314 +0x174
github.com/boltdb/bolt.(*Cursor).search(0xc820367490, 0x25840030, 0x9b, 0x7fffffff, 0x2538a)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cursor.go:271 +0x3aa
github.com/boltdb/bolt.(*Cursor).seek(0xc820367490, 0x25840030, 0x9b, 0x7fffffff, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cursor.go:159 +0xda
github.com/boltdb/bolt.(*Bucket).Bucket(0xc820074100, 0x25840030, 0x9b, 0x7fffffff, 0x7fffffff)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/bucket.go:112 +0x180
github.com/boltdb/bolt.(*Tx).checkBucket.func2(0x25840030, 0x9b, 0x7fffffff, 0x258400cb, 0x273, 0x7fffffff, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:403 +0x78
github.com/boltdb/bolt.(*Bucket).ForEach(0xc820074100, 0xc8203675f0, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/bucket.go:361 +0x175
github.com/boltdb/bolt.(*Tx).checkBucket(0xc8200a6000, 0xc820074100, 0xc820367e88, 0xc820367eb8, 0xc82001a120)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:407 +0x101
github.com/boltdb/bolt.(*Tx).checkBucket.func2(0x25862046, 0x8, 0x7fffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:404 +0xb5
github.com/boltdb/bolt.(*Bucket).ForEach(0xc8200123c0, 0xc820367790, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/bucket.go:361 +0x175
github.com/boltdb/bolt.(*Tx).checkBucket(0xc8200a6000, 0xc8200123c0, 0xc820367e88, 0xc820367eb8, 0xc82001a120)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:407 +0x101
github.com/boltdb/bolt.(*Tx).checkBucket.func2(0x2586406a, 0x7, 0x7fffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:404 +0xb5
github.com/boltdb/bolt.(*Bucket).ForEach(0xc8200740c0, 0xc820367930, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/bucket.go:361 +0x175
github.com/boltdb/bolt.(*Tx).checkBucket(0xc8200a6000, 0xc8200740c0, 0xc820367e88, 0xc820367eb8, 0xc82001a120)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:407 +0x101
github.com/boltdb/bolt.(*Tx).checkBucket.func2(0x25871046, 0x8, 0x7fffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:404 +0xb5
github.com/boltdb/bolt.(*Bucket).ForEach(0xc820012300, 0xc820367ad0, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/bucket.go:361 +0x175
github.com/boltdb/bolt.(*Tx).checkBucket(0xc8200a6000, 0xc820012300, 0xc820367e88, 0xc820367eb8, 0xc82001a120)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:407 +0x101
github.com/boltdb/bolt.(*Tx).checkBucket.func2(0x2587406a, 0x7, 0x7fffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:404 +0xb5
github.com/boltdb/bolt.(*Bucket).ForEach(0xc8200a6018, 0xc820367c70, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/bucket.go:361 +0x175
github.com/boltdb/bolt.(*Tx).checkBucket(0xc8200a6000, 0xc8200a6018, 0xc820367e88, 0xc820367eb8, 0xc82001a120)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:407 +0x101
github.com/boltdb/bolt.(*Tx).check(0xc8200a6000, 0xc82001a120)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:358 +0x708
created by github.com/boltdb/bolt.(*Tx).Check
    /Users/di/Projects/go/src/github.com/boltdb/bolt/tx.go:335 +0x5d

goroutine 1 [chan receive]:
main.(*CheckCommand).Run.func1(0xc8200a6000, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cmd/bolt/main.go:192 +0xb2
github.com/boltdb/bolt.(*DB).View(0xc8200a2000, 0xc820043b70, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/db.go:583 +0xb9
main.(*CheckCommand).Run(0xc820043de8, 0xc82007a260, 0x1, 0x1, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cmd/bolt/main.go:210 +0x483
main.(*Main).Run(0xc820043f18, 0xc82007a250, 0x2, 0x2, 0x0, 0x0)
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cmd/bolt/main.go:104 +0xa97
main.main()
    /Users/di/Projects/go/src/github.com/boltdb/bolt/cmd/bolt/main.go:64 +0x15e
@benbjohnson
Copy link
Member

Can you give me some more details about your OS & application?

@k0mrade
Copy link
Author

k0mrade commented Aug 1, 2016

Sorry for the delay.

Operating System: Debian GNU/Linux 8 (Jessie)
Kernel: Linux 3.16.0-4-amd64
Architecture: x86-64

App
It's WiFi Hotspot solution. BoltDB uses for store redirect URL (WiFi sessions), users data and accounting etc.

Now this app works on two sites. We work with Ruckus controller and 50 WiFi access points.
Ruckus controller works as a network access server and it has Northbound interface that implements Login/Logout etc REST API.

It's the concept app, but I'm thinking to use BoltDB as the main database solution in future.

What is the best way to create an index with BoltDB? I saw github.com/asdine/storm and github.com/tidwall/buntdb implementations. The first one is the copy to another bucket solution.
The second one is Btree key copy, I think.

Thank you.

@benbjohnson
Copy link
Member

@k0mrade re: indexes, I typically use a separate bucket to store the index. For example, if you have a bucket of Widgets then you could make a separate Widgets.Name bucket that stores name => key index. That works for unique indexes.

If you need non-unique indexes you can use the index bucket to point a value to a subbucket where there are a list of keys to point to.

Depending on your cardinality of data you can also embed the index in the original bucket. For example, if you know you'll only have a small number of users per account you can embed a list of user IDs in your Account object. There's a lot of ways to do indexes in Bolt.

@asdine
Copy link
Contributor

asdine commented Aug 6, 2016

@k0mrade Yep what @benbjohnson said is exactly what has been done in Storm. Using buckets as indexes to other buckets is a very easy solution and works very well.

I tried to implement a Btree but the encoding and decoding part is way too slow.

The BoltDB buckets already do the job for you as you have a O(1) time complexity on index access out of the box for unique indexes.
You just need to create a bucket and store your value as the key, and the ID as the value:

(unique index bucket)
  value1 => ID1
  value2 => ID2

If you need non-unique indexes, create a sub bucket with IDs as keys and nothing as the value:

(list index bucket)
  value1 =>
    (sub bucket)
      ID1 => null
      ID2 => null
      ID3 => null
  value2 =>
    (sub bucket)
      ID9 => null
      ID4 => null
      ID1 => null

Very easy to implement.

Hope that helps

@tidwall tidwall mentioned this issue Oct 12, 2016
Open
@rudibs rudibs mentioned this issue Jun 11, 2017
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@benbjohnson @k0mrade @asdine