Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: agent panics on arm #831

Merged
merged 1 commit into from Oct 10, 2018
Merged

fix: agent panics on arm #831

merged 1 commit into from Oct 10, 2018

Conversation

jhedev
Copy link
Contributor

@jhedev jhedev commented Oct 9, 2018

When running the latest version on arm systems I get the following:

  _           _ _     _ _    _ _                                _
 | |         (_) |   | | |  (_) |                              | |
 | |__  _   _ _| | __| | | ___| |_ ___    __ _  __ _  ___ _ __ | |_
 | '_ \| | | | | |/ _` | |/ / | __/ _ \  / _` |/ _` |/ _ \ '_ \| __|
 | |_) | |_| | | | (_| |   <| | ||  __/ | (_| | (_| |  __/ | | | |_
 |_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___|  \__,_|\__, |\___|_| |_|\__|
                                                __/ |
 http://buildkite.com/agent                    |___/

2018-10-09 08:29:05 NOTICE Starting buildkite-agent v3.5.2 with PID: 16
2018-10-09 08:29:05 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
2018-10-09 08:29:05 NOTICE For questions and support, email us at: hello@buildkite.com
2018-10-09 08:29:05 WARN   Failed to find unique machine-id: machineid: machineid: open /etc/machine-id: no such file or directory
2018-10-09 08:29:05 INFO   Registering agent with Buildkite...
2018-10-09 08:29:07 INFO   Successfully registered agent "8217c7dc4b66" with tags [queue=armv7]
2018-10-09 08:29:07 INFO   Connecting to Buildkite...
2018-10-09 08:29:07 INFO   Agent successfully connected
2018-10-09 08:29:07 INFO   You can press Ctrl-C to stop the agent
2018-10-09 08:29:07 INFO   Waiting for work...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0x114fc]

goroutine 28 [running]:
sync/atomic.storeUint64(0x1144e03c, 0x5bbc66d3, 0x0)
	/usr/local/go/src/sync/atomic/64bit_arm.go:20 +0x3c
github.com/buildkite/agent/agent.(*AgentWorker).Heartbeat(0x1144e000, 0x0, 0x0)
	/go/src/github.com/buildkite/agent/agent/agent_worker.go:228 +0x108
github.com/buildkite/agent/agent.(*AgentWorker).Start.func1(0x1144e000, 0xf8475800, 0xd)
	/go/src/github.com/buildkite/agent/agent/agent_worker.go:78 +0x44
created by github.com/buildkite/agent/agent.(*AgentWorker).Start
	/go/src/github.com/buildkite/agent/agent/agent_worker.go:75 +0x68

We have seen this issue with other golang projects on arm in the past as well (see for example dgraph-io/badger#311). This is caused by alignment issues that panic when using atomic.StoreInt64.

I tested this fix on an ARM machine and it works fine now.

This is caused by an alignment issue when using atomic.StoreInt64.

Moving the Int64 values to the top of the struct fixes it.
Copy link
Member

@keithpitt keithpitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow, that's crazy hey! Looks good to me, I'll let @lox merge and push out a release though.

@lox
Copy link
Contributor

lox commented Oct 10, 2018

Thanks for this @jhedev!

@lox lox merged commit cc07aba into buildkite:master Oct 10, 2018
@jhedev jhedev deleted the fix/arm_atomic_panic branch October 10, 2018 05:51
@jhedev
Copy link
Contributor Author

jhedev commented Oct 10, 2018

Thanks for the quick merge guys. Yeah, it's really crazy... Especially, if you don't know about it :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants