Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

race condition when running ipfs daemon #7952

Open
marten-seemann opened this issue Feb 27, 2021 · 1 comment
Open

race condition when running ipfs daemon #7952

marten-seemann opened this issue Feb 27, 2021 · 1 comment
Labels
kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization

Comments

@marten-seemann
Copy link
Member

marten-seemann commented Feb 27, 2021 •
edited

Version information:

go-ipfs version: 0.8.0
Repo version: 11
System version: amd64/darwin
Golang version: go1.16

Description:

This is easy to reproduce: go build -race cmd/ipfs && cmd/ipfs/ipfs. The race condition occurs less than 20s after booting the node.

WARNING: DATA RACE
Read at 0x00c000776348 by goroutine 309:
  github.com/ipfs/go-ipfs/commands.(*Context).GetNode()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/commands/context.go:59 +0x4a
  github.com/ipfs/go-ipfs/commands.(*Context).Context()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/commands/context.go:101 +0x55
  main.startPinMFS.func1()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/pinmfs.go:75 +0xdb

Previous write at 0x00c000776348 by goroutine 210:
  github.com/ipfs/go-ipfs/commands.(*Context).GetNode()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/commands/context.go:63 +0x12f
  github.com/ipfs/go-ipfs/commands.(*Context).Context()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/commands/context.go:101 +0x55
  main.pinMFSOnChange()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/pinmfs.go:101 +0x168

Goroutine 309 (running) created at:
  main.startPinMFS()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/pinmfs.go:67 +0xda
  main.daemonFunc()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/daemon.go:443 +0x16a4
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  github.com/syndtr/goleveldb/leveldb.(*DB).checkAndCleanFiles()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_util.go:52 +0x301
  github.com/syndtr/goleveldb/leveldb.openDB()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:130 +0x9bb
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x284
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:657 +0x19e
  fmt.(*ss).doScanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:1230 +0x411
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:476 +0xc8
  github.com/syndtr/goleveldb/leveldb.openDB()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:125 +0x997
  github.com/syndtr/goleveldb/leveldb.Open()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:197 +0x25a
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.fsParseNamePtr()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:666 +0x7ac
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).GetMeta.func1()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:332 +0x140
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).GetMeta.func2()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:356 +0xd7
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).GetMeta()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:407 +0x753
  github.com/syndtr/goleveldb/leveldb.(*session).recover()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/session.go:119 +0xfe
  github.com/syndtr/goleveldb/leveldb.Open()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:183 +0x109
  github.com/syndtr/goleveldb/leveldb.OpenFile()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:219 +0xbb
  github.com/ipfs/go-ds-leveldb.NewDatastore()
      /Users/marten/src/go/pkg/mod/github.com/ipfs/go-ds-leveldb@v0.4.2/datastore.go:46 +0x40c
  github.com/ipfs/go-ipfs/plugin/plugins/levelds.(*datastoreConfig).Create()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/plugin/plugins/levelds/levelds.go:85 +0xfa
  github.com/ipfs/go-ipfs/repo/fsrepo.measureDatastoreConfig.Create()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/datastores.go:243 +0x5c
  github.com/ipfs/go-ipfs/repo/fsrepo.(*measureDatastoreConfig).Create()
      <autogenerated>:1 +0xac
  github.com/ipfs/go-ipfs/repo/fsrepo.(*mountDatastoreConfig).Create()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/datastores.go:154 +0x13e
  github.com/ipfs/go-ipfs/repo/fsrepo.(*FSRepo).openDatastore()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:440 +0x4aa
  github.com/ipfs/go-ipfs/repo/fsrepo.open()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:169 +0x476
  github.com/ipfs/go-ipfs/repo/fsrepo.Open.func1()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:113 +0x4e
  github.com/ipfs/go-ipfs/repo.(*OnlyOne).Open()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/onlyone.go:35 +0x207
  github.com/ipfs/go-ipfs/repo/fsrepo.Open()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:115 +0x9c
  main.daemonFunc()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/daemon.go:273 +0x388
  github.com/ipfs/go-ipfs-cmds.(*executor).Execute()
      /Users/marten/src/go/pkg/mod/github.com/ipfs/go-ipfs-cmds@v0.6.0/executor.go:77 +0x227
  github.com/ipfs/go-ipfs-cmds/cli.Run()
      /Users/marten/src/go/pkg/mod/github.com/ipfs/go-ipfs-cmds@v0.6.0/cli/run.go:137 +0xec6
  main.mainRet()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:168 +0x434
  main.main()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:71 +0x2f

Goroutine 210 (running) created at:
  main.startPinMFS()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/pinmfs.go:66 +0xa4
  main.daemonFunc()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/daemon.go:443 +0x16a4
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  github.com/syndtr/goleveldb/leveldb.(*DB).checkAndCleanFiles()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db_util.go:52 +0x301
  github.com/syndtr/goleveldb/leveldb.openDB()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:130 +0x9bb
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x284
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:657 +0x19e
  fmt.(*ss).doScanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:1230 +0x411
  fmt.Fscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:143 +0xee
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:458 +0x309
  github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:476 +0xc8
  github.com/syndtr/goleveldb/leveldb.openDB()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:125 +0x997
  github.com/syndtr/goleveldb/leveldb.Open()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:197 +0x25a
  fmt.Sscanf()
      /usr/local/Cellar/go/1.16/libexec/src/fmt/scan.go:114 +0x191
  github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:643 +0xa6
  github.com/syndtr/goleveldb/leveldb/storage.fsParseNamePtr()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:666 +0x7ac
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).GetMeta.func1()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:332 +0x140
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).GetMeta.func2()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:356 +0xd7
  github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).GetMeta()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/storage/file_storage.go:407 +0x753
  github.com/syndtr/goleveldb/leveldb.(*session).recover()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/session.go:119 +0xfe
  github.com/syndtr/goleveldb/leveldb.Open()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:183 +0x109
  github.com/syndtr/goleveldb/leveldb.OpenFile()
      /Users/marten/src/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.0/leveldb/db.go:219 +0xbb
  github.com/ipfs/go-ds-leveldb.NewDatastore()
      /Users/marten/src/go/pkg/mod/github.com/ipfs/go-ds-leveldb@v0.4.2/datastore.go:46 +0x40c
  github.com/ipfs/go-ipfs/plugin/plugins/levelds.(*datastoreConfig).Create()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/plugin/plugins/levelds/levelds.go:85 +0xfa
  github.com/ipfs/go-ipfs/repo/fsrepo.measureDatastoreConfig.Create()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/datastores.go:243 +0x5c
  github.com/ipfs/go-ipfs/repo/fsrepo.(*measureDatastoreConfig).Create()
      <autogenerated>:1 +0xac
  github.com/ipfs/go-ipfs/repo/fsrepo.(*mountDatastoreConfig).Create()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/datastores.go:154 +0x13e
  github.com/ipfs/go-ipfs/repo/fsrepo.(*FSRepo).openDatastore()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:440 +0x4aa
  github.com/ipfs/go-ipfs/repo/fsrepo.open()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:169 +0x476
  github.com/ipfs/go-ipfs/repo/fsrepo.Open.func1()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:113 +0x4e
  github.com/ipfs/go-ipfs/repo.(*OnlyOne).Open()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/onlyone.go:35 +0x207
  github.com/ipfs/go-ipfs/repo/fsrepo.Open()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/repo/fsrepo/fsrepo.go:115 +0x9c
  main.daemonFunc()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/daemon.go:273 +0x388
  github.com/ipfs/go-ipfs-cmds.(*executor).Execute()
      /Users/marten/src/go/pkg/mod/github.com/ipfs/go-ipfs-cmds@v0.6.0/executor.go:77 +0x227
  github.com/ipfs/go-ipfs-cmds/cli.Run()
      /Users/marten/src/go/pkg/mod/github.com/ipfs/go-ipfs-cmds@v0.6.0/cli/run.go:137 +0xec6
  main.mainRet()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:168 +0x434
  main.main()
      /Users/marten/src/go/src/github.com/ipfs/go-ipfs/cmd/ipfs/main.go:71 +0x2f
==================
@marten-seemann marten-seemann added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Feb 27, 2021
@Stebalien
Copy link
Member

Probably one of those races now caught by go 1.16.

gammazero added a commit that referenced this issue Mar 4, 2021
@gammazero
Fix race condition with node creation
1e9ceec 
This fixes issue #7952
@gammazero gammazero mentioned this issue Mar 4, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Stebalien @marten-seemann