Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic: unaligned 64-bit atomic operation on 32-bit arm host #6944

Open
abferm opened this issue Aug 19, 2022 · 3 comments
Open

panic: unaligned 64-bit atomic operation on 32-bit arm host #6944

abferm opened this issue Aug 19, 2022 · 3 comments

Comments

@abferm
Copy link
Contributor

abferm commented Aug 19, 2022

Describe the bug
Panic from ingester on 32-bit arm hosts due to misaligned 64-bit atomic operation.

To Reproduce
Steps to reproduce the behavior:

  1. Started grafana/loki:2.6.1-arm docker image
  2. Started feeding logs with promtail

Expected behavior
No panic

Environment:

  • Infrastructure: balena OS on 32-bit arm device
  • Deployment tool: balena

Screenshots, Promtail config, or terminal output

 panic: unaligned 64-bit atomic operation
 
 goroutine 507 [running]:
 runtime/internal/atomic.panicUnaligned()
  /usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x24
 runtime/internal/atomic.Xadd64(0x4cc88cc, 0x1)
  /usr/local/go/src/runtime/internal/atomic/atomic_arm.s:256 +0x14
 github.com/grafana/loki/pkg/logqlmodel/stats.(*Context).AddHeadChunkLines(...)
  /src/loki/pkg/logqlmodel/stats/context.go:225
 github.com/grafana/loki/pkg/chunkenc.(*unorderedHeadBlock).forEntries.func1(0x44a7dd0)
  /src/loki/pkg/chunkenc/unordered.go:169 +0x54
 github.com/grafana/loki/pkg/chunkenc.(*unorderedHeadBlock).forEntries(0x48b37c0, {0x1f85088, 0x4446400}, 0x0, 0x0, 0x7fffffffffffffff, 0x4b15ccc)
  /src/loki/pkg/chunkenc/unordered.go:194 +0x2a8
 github.com/grafana/loki/pkg/chunkenc.(*unorderedHeadBlock).Serialise(0x48b37c0, {0x76c918a8, 0x2e26858})
  /src/loki/pkg/chunkenc/unordered.go:335 +0x184
 github.com/grafana/loki/pkg/chunkenc.(*MemChunk).cut(0x4904810)
  /src/loki/pkg/chunkenc/memchunk.go:744 +0x130
 github.com/grafana/loki/pkg/chunkenc.(*MemChunk).Close(0x4904810)
  /src/loki/pkg/chunkenc/memchunk.go:692 +0x1c
 github.com/grafana/loki/pkg/ingester.(*Ingester).closeChunk(0x481f180, 0x49e0200, {0x1f7f080, 0x4b84b24})
  /src/loki/pkg/ingester/flush.go:340 +0x74
 github.com/grafana/loki/pkg/ingester.(*Ingester).flushChunks(0x481f180, {0x1f850a8, 0x49e02c0}, 0x1e84eee75f66f89d, {0x48b36c0, 0x2, 0x2}, {0x449fe58, 0x1, 0x2}, ...)
  /src/loki/pkg/ingester/flush.go:290 +0x31c
 github.com/grafana/loki/pkg/ingester.(*Ingester).flushUserSeries(0x481f180, {0x4ad6328, 0x4}, 0x1e84eee75f66f89d, 0x1)
  /src/loki/pkg/ingester/flush.go:172 +0x1c8
 github.com/grafana/loki/pkg/ingester.(*Ingester).flushLoop(0x481f180, 0x1d)
  /src/loki/pkg/ingester/flush.go:144 +0x2d4
 created by github.com/grafana/loki/pkg/ingester.(*Ingester).InitFlushQueues
  /src/loki/pkg/ingester/flush.go:47 +0x4c
@frepkovsky
Copy link

I think it is the same problem as discussed here #5388 but it has already been closed without solution...

@abferm
Copy link
Contributor Author

abferm commented Aug 23, 2022

This does appear to be the same issue. I thought I searched the closed issues, but obviously I didn't look hard enough.

@gtaws
Copy link

gtaws commented Oct 16, 2022

as commented on the linked issue, can someone PLEASE REMOVE loki-linux-arm.zip, as it's obviously broken now with no resolution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants