runtime error: invalid memory address or nil pointer dereference #165

twolao · 2022-05-25T23:41:33Z

Describe the bug
A clear and concise description of what the bug is.

runtime error: invalid memory address or nil pointer dereference
/usr/local/go/src/runtime/panic.go:220 (0x404c295)
panicmem: panic(memoryError)
/usr/local/go/src/runtime/signal_unix.go:818 (0x404c265)
sigpanic: panicmem()
//pkg/mod/github.com/xujiajun/nutsdb@v0.8.0/tx.go:123 (0x48d6518)
(Tx).getTxID: node, err := snowflake.NewNode(tx.db.opt.NodeNum)
//pkg/mod/github.com/xujiajun/nutsdb@v0.8.0/tx.go:111 (0x48d649e)
newTx: txID, err = tx.getTxID()
/*/pkg/mod/github.com/xujiajun/nutsdb@v0.8.0/tx.go:85 (0x48d6364)
(*DB).Begin: tx, err = newTx(db, writable)
/*r/pkg/mod/github.com/xujiajun/nutsdb@v0.8.0/db.go:936 (0x48d5184)
(DB).managed: tx, err := db.Begin(writable)
//pkg/mod/github.com/xujiajun/nutsdb@v0.8.0/db.go:293 (0x4969490)
(*DB).View: return db.managed(false, fn)

To Reproduce
Steps to reproduce the behavior(Be specific!):

由于程序退出导致再次启动后每次read一个key时报错，退出重启无法恢复，必须删除本地的nutsdb文件才可以正常运行。
程序平时可以正常运行，无法确定什么时间退出会导致这个异常，异常出现后除了删除文件外无法自动恢复
错误出现在

Give sample code if you can.

func read(k1 string) (string, error) {
    var out string
    if err := db.View(
        func(tx *nutsdb.Tx) error {
            key := []byte(k1)
            e, err := tx.Get(bucket, key)
            if err != nil {
                return err
            }
            out = string(e.Value)

            return nil
        }); err != nil {

        return "", err
    } 
        
    return out, nil
}

Expected behavior
A clear and concise description of what you expected to happen.
key不存在不会导致异常panic，可以返回错误信息进行处理

What actually happens
A clear and concise description of what actually happens.

Screenshots
If applicable, add screenshots to help explain your problem.

please complete the following information :

OS: [e.g. Ubuntu 16.04]
Mac OSX
NutsDB Version [e.g. 0.4.0]
github.com/xujiajun/nutsdb v0.8.0

Additional context
Add any other context about the problem here.

xujiajun · 2022-05-26T12:38:47Z

@Oceanchang What is the configuration of your nutsdb startup? Can you give me a complete code that you can remove the business code and reproduce the problem?

twolao · 2022-05-27T02:41:28Z

@Oceanchang What is the configuration of your nutsdb startup? Can you give me a complete code that you can remove the business code and reproduce the problem?

jukanntenn · 2022-05-28T05:31:56Z

如果方便的话，可否直接使用 Markdown 语法粘贴代码，这样方便复制粘贴代码到本地运行，否则只能照着代码一个个手输。

jukanntenn · 2022-05-28T05:59:52Z

我尝试本地运行了一下，没有 panic，这是测试的代码（截图中的代码无法直接运行，因此贴出的代码在截图中代码的基础上做了一定修改）：

package main

import (
	"fmt"

	"github.com/xujiajun/nutsdb"
)

var bucket string
var db *nutsdb.DB

func init() {
	opt := nutsdb.DefaultOptions
	fileDir := "./nutsdb"
	opt.Dir = fileDir
	opt.SegmentSize = 1024 * 1024
	db, _ = nutsdb.Open(opt)
	bucket = "bucketForString"
}

func main() {
	username := "test"
	pid, err := read(username + "_pid")

	fmt.Println(pid, "---", err)
         // 输出结果：--- bucket not found:bucket:bucketForString,key:test_pid
}

func read(k1 string) (string, error) {
	var out string
	if err := db.View(
		func(tx *nutsdb.Tx) error {
			key := []byte(k1)
			e, err := tx.Get(bucket, key)
			if err != nil {
				return err
			}
			out = string(e.Value)

			return nil
		}); err != nil {

		return "", err
	}

	return out, nil
}

运行环境：MacOS，go 1.17，nutsdb@v0.8.0

twolao · 2022-05-28T07:06:34Z

如果方便的话，可否直接使用 Markdown 语法粘贴代码，这样方便复制粘贴代码到本地运行，否则只能照着代码一个个手输。

谢谢，忽然想到是否是因为没有defer db.Close() 导致有时程序退出时错误，所以在每次read时执行db的open和close，使用曾出现异常的文件运行时，报错如下：

2022/05/28 15:05:01 db.buildIndexes error: when build activeDataIndex readAt err: crc error
exit status 1

func read(k1 string) (string, error) {
    db, err = nutsdb.Open(opt)
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    var out string
    
    if err := db.View(
        func(tx *nutsdb.Tx) error {
            key := []byte(k1)
            e, err := tx.Get(bucket, key)
            if err != nil {
                return err
            }
            out = string(e.Value)

            return nil
        }); err != nil {

        return "", err
    } 
        
    return out, nil
    
}

xujiajun · 2022-05-29T14:12:05Z

如果方便的话，可否直接使用 Markdown 语法粘贴代码，这样方便复制粘贴代码到本地运行，否则只能照着代码一个个手输。

谢谢，忽然想到是否是因为没有defer db.Close() 导致有时程序退出时错误，所以在每次read时执行db的open和close，使用曾出现异常的文件运行时，报错如下：

2022/05/28 15:05:01 db.buildIndexes error: when build activeDataIndex readAt err: crc error exit status 1
func read(k1 string) (string, error) {
    db, err = nutsdb.Open(opt)
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    var out string
    
    if err := db.View(
        func(tx *nutsdb.Tx) error {
            key := []byte(k1)
            e, err := tx.Get(bucket, key)
            if err != nil {
                return err
            }
            out = string(e.Value)

            return nil
        }); err != nil {

        return "", err
    } 
        
    return out, nil
    
}

Open 是个很重的动作，你可以在main里面初始化的一个db实例就好了，不用每次read都open 和 close，后面用这个db实例。
crc error错误说明你数据有异常。

twolao · 2022-06-01T00:46:20Z

如果方便的话，可否直接使用 Markdown 语法粘贴代码，这样方便复制粘贴代码到本地运行，否则只能照着代码一个个手输。

谢谢，忽然想到是否是因为没有defer db.Close() 导致有时程序退出时错误，所以在每次read时执行db的open和close，使用曾出现异常的文件运行时，报错如下：
2022/05/28 15:05:01 db.buildIndexes error: when build activeDataIndex readAt err: crc error exit status 1
func read(k1 string) (string, error) {
    db, err = nutsdb.Open(opt)
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    var out string
    
    if err := db.View(
        func(tx *nutsdb.Tx) error {
            key := []byte(k1)
            e, err := tx.Get(bucket, key)
            if err != nil {
                return err
            }
            out = string(e.Value)

            return nil
        }); err != nil {

        return "", err
    } 
        
    return out, nil
    
}
Open 是个很重的动作，你可以在main里面初始化的一个db实例就好了，不用每次read都open 和 close，后面用这个db实例。 crc error错误说明你数据有异常。

之前就是有初始化一个db这样使用，结果是有时候程序killed，再次起来就报标题的错误了，如果遇到open提示crc error是否可以在程序层面处理？

xujiajun · 2022-06-01T12:13:08Z

crc error 是数据有异常，不知道你是怎么搞出来的这个错误，一般不会发生，程序层面貌似不能处理，首先要弄清楚什么情况下出现了crc error

twolao · 2022-06-02T00:15:57Z

crc error 是数据有异常，不知道你是怎么搞出来的这个错误，一般不会发生，程序层面貌似不能处理，首先要弄清楚什么情况下出现了crc error

知道问题出在什么地方了，有可能是多个地方open这个文件，或反复open导致的，集中到一个package中做初始化打开一个db应该可以避免这个问题再次发生，thanx

yingziwu · 2022-12-28T10:34:15Z

遇到了同样的问题

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1b7b798]
goroutine 3633 [running]:
github.com/xujiajun/nutsdb.(*Tx).getTxID(0x10c00c009cd7a20?)
        github.com/xujiajun/nutsdb@v0.11.1/tx.go:123 +0x18
github.com/xujiajun/nutsdb.newTx(0x0, 0x0)
        github.com/xujiajun/nutsdb@v0.11.1/tx.go:111 +0x9f
github.com/xujiajun/nutsdb.(*DB).Begin(0x0, 0xe8?)
        github.com/xujiajun/nutsdb@v0.11.1/tx.go:85 +0x25
github.com/xujiajun/nutsdb.(*DB).managed(0xc004c0ea80?, 0x80?, 0xc005aedbf8)
        github.com/xujiajun/nutsdb@v0.11.1/db.go:978 +0x39
github.com/xujiajun/nutsdb.(*DB).View(...)
        github.com/xujiajun/nutsdb@v0.11.1/db.go:307
github.com/darkweak/souin/cache/providers.(*Nuts).Prefix(0xc005aedd88?, {0xc000bdb6d0?, 0x2422cc6?}, 0xf?)
        github.com/darkweak/souin@v1.6.27/cache/providers/nutsProvider.go:143 +0x72
github.com/darkweak/souin/rfc.CachedResponse({0x29c2638, 0xc000aa6330}, 0xc009ece300, {0xc000bdb6d0, 0x48}, {0x43ce60?, 0xc004648d00?})
        github.com/darkweak/souin@v1.6.27/rfc/bridge.go:25 +0x1fb
github.com/darkweak/souin/plugins.DefaultSouinPluginCallback.func1(0x40?, {0x29c6548, 0xc000968360}, {0xc000bdb6d0, 0x48})
        github.com/darkweak/souin@v1.6.27/plugins/base.go:148 +0xf8
created by github.com/darkweak/souin/plugins.DefaultSouinPluginCallback
        github.com/darkweak/souin@v1.6.27/plugins/base.go:146 +0x4e5

使用的是 Caddy cache 插件，cache 部分的配置文件如下，在中等流量条件下几钟内就会 panic 退出。

{
      "log_level": "WARN",
      "allowed_http_verbs": ["GET", "HEAD"],
      "headers": ["Content-Type", "Authorization"],
      "key": {
          "disable_host": true,
          "disable_body": true,
          "disable_method": false,
          "headers": ["Content-Type"]
      },
      "nuts": {
          "configuration": {
              "Dir": "/var/cache/caddy",
              "EntryIdxMode": "HintKeyAndRAMIdxMode",
              "RWMode": "FileIO",
              "SegmentSize": "256MB",
              "NodeNum": 42,
              "SyncEnable": false,
              "StartFileLoadingMode": "MMap"
          }
      },
      "ttl": "7200s"
}

twolao assigned xujiajun May 25, 2022

twolao closed this as completed Jun 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime error: invalid memory address or nil pointer dereference #165

runtime error: invalid memory address or nil pointer dereference #165

twolao commented May 25, 2022 •

edited by xujiajun

Loading

xujiajun commented May 26, 2022

twolao commented May 27, 2022 •

edited

Loading

jukanntenn commented May 28, 2022

jukanntenn commented May 28, 2022

twolao commented May 28, 2022 •

edited by xujiajun

Loading

xujiajun commented May 29, 2022

twolao commented Jun 1, 2022

xujiajun commented Jun 1, 2022

twolao commented Jun 2, 2022

yingziwu commented Dec 28, 2022

runtime error: invalid memory address or nil pointer dereference #165

runtime error: invalid memory address or nil pointer dereference #165

Comments

twolao commented May 25, 2022 • edited by xujiajun Loading

xujiajun commented May 26, 2022

twolao commented May 27, 2022 • edited Loading

jukanntenn commented May 28, 2022

jukanntenn commented May 28, 2022

twolao commented May 28, 2022 • edited by xujiajun Loading

xujiajun commented May 29, 2022

twolao commented Jun 1, 2022

xujiajun commented Jun 1, 2022

twolao commented Jun 2, 2022

yingziwu commented Dec 28, 2022

twolao commented May 25, 2022 •

edited by xujiajun

Loading

twolao commented May 27, 2022 •

edited

Loading

twolao commented May 28, 2022 •

edited by xujiajun

Loading