Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protect object map from concurrent access #340

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Conversation

ubkr
Copy link

@ubkr ubkr commented Oct 27, 2022

In situations with many concurrent dbus requests and at the same time new objects are registered it is possible to end up in a situation where the object map is write and read accessed from different go routines resulting in a panic.

The patch is working in our setup but I've not run any test other than 'go test'

Copy link
Member

@guelfey guelfey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, some small comments on the fix implementation

export.go Outdated
@@ -396,7 +395,7 @@ func (conn *Conn) export(methods map[string]reflect.Value, path ObjectPath, ifac

// If this is the first handler for this path, make a new map to hold all
// handlers for this path.
if !h.PathExists(path) {
if _, ok := h.PathExists(path); !ok {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better get a write lock here then and keep it until the end of the function - this also removes the need for checking the other PathExists in L408.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main problem I can see with that is what lock should be taken, I would argue that it is not a good design to call h.Lock() from this context and if we choose to do that we need to remove the lock call in h.AddObject too, making it asymmetric to h.RemoveObject. To me it is better to let defaultHandler and its functions handle its own locks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the way this is implemented right now introduces a race condition. If you have two export calls running at the same time for the same path, it could happen that:

  1. Call 1 reaches export.go:398. There's no object on that path yet, so ok is false. After GetExportedObject returns, the lock is not held.
  2. Call 2 goes through the whole function, working on the newly created entry in h.objects.
  3. Call 1 is scheduled again and reaches export.go:399, overwriting the entry in h.objects and thus discarding the effect that Call 2 had.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(sorry about the delay..)
@guelfey, Added a new function to defaultHandler, GetAnExportedObject(path). This function will return either an existing object for the path or register a newly created object on the path and return it. It should address your concerns above but I'm not sure that it is a good name though.
Note, I've yet to run it through our own use cases to make sure it works....

default_handler.go Outdated Show resolved Hide resolved
ubkr and others added 3 commits November 10, 2022 14:22
GetAnExportedObject returns either an existing object or a newly
created object for a specific path. This will solve the case where
we could get a mixup when two different goroutines competed to
create an object for a specific path. The result might still be
unsatifying but atleast its either or and not a mix.
Comment on lines +51 to +52
h.RLock()
defer h.RUnlock()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be a write lock then. The race detector picked this up as well: https://github.com/godbus/dbus/actions/runs/4797626686/jobs/9033883115?pr=340#step:6:243

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So true. Fixed!

default_handler.go Outdated Show resolved Hide resolved
default_handler.go Outdated Show resolved Hide resolved
ubkr and others added 3 commits May 30, 2023 15:42
GetOrAddExportedObject returns either an existing object or a newly
created object for a specific path. This will solve the case where
we could get a mixup when two different goroutines competed to
create an object for a specific path. The result might still be
unsatifying but atleast its either or and not a mix.
@ubkr
Copy link
Author

ubkr commented Jul 14, 2023

@guelfey Hi, would it be possible to move this forward or do you think there is more needed by me?

Comment on lines +44 to +45
obj, ok := h.objects[path]
return obj, ok

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
obj, ok := h.objects[path]
return obj, ok
return h.objects[path]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants