Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bridge-device: fix concurrent map writes #470

Merged
merged 1 commit into from
May 22, 2024

Conversation

jkralik
Copy link
Member

@jkralik jkralik commented May 22, 2024

fatal error: concurrent map writes

goroutine 263 [running]:
github.com/plgd-dev/device/v2/bridge/device/thingDescription.PatchThingDescription.func1({0x88234a, 0xd}, {0x9382d0?, 0xc000184c98?})
	/go/src/github.com/plgd-dev/device/bridge/device/thingDescription/thingDescription.go:231 +0x145
github.com/plgd-dev/go-coap/v3/pkg/sync.(*Map[...]).Range(0x824840, 0xc001139368?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/pkg/sync/map.go:111 +0x166
github.com/plgd-dev/device/v2/bridge/device.(*Device).Range(0xc00102b0b0?, 0xc00024f2bc?)
	/go/src/github.com/plgd-dev/device/bridge/device/device.go:235 +0x25
github.com/plgd-dev/device/v2/bridge/device/thingDescription.PatchThingDescription({0xc0001d5660, 0xc0001d5680, 0x0, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, 0x0}, ...}, ...}, ...)
	/go/src/github.com/plgd-dev/device/bridge/device/thingDescription/thingDescription.go:223 +0x388
main.getTDOpts.func1({0x76e708?, 0xc0001d5820?}, 0xc0001d3ba0, {0x0?, 0xc001050600?, 0x2?})
	/go/src/github.com/plgd-dev/device/cmd/bridge-device/main.go:118 +0x125
github.com/plgd-dev/device/v2/bridge/device.New.func5({0x9358e8?, 0xc0001d8870?}, {0x0?, 0x773667?, 0x0?})
	/go/src/github.com/plgd-dev/device/bridge/device/device.go:181 +0x3b
github.com/plgd-dev/device/v2/bridge/resources/thingDescription.(*Resource).Get(0xc000199110, 0xc000186660)
	/go/src/github.com/plgd-dev/device/bridge/resources/thingDescription/resource.go:88 +0x43
github.com/plgd-dev/device/v2/bridge/resources.(*Resource).observerHandler(0xc00027e150, 0xc000186660, 0xf8?)
	/go/src/github.com/plgd-dev/device/bridge/resources/resource.go:286 +0x138
github.com/plgd-dev/device/v2/bridge/resources.(*Resource).HandleRequest(0xc00027e150, 0xc000186660)
	/go/src/github.com/plgd-dev/device/bridge/resources/resource.go:317 +0x8d
github.com/plgd-dev/device/v2/bridge/device.(*Device).HandleRequest(0xc0001d3ba0, 0xc000186660)
	/go/src/github.com/plgd-dev/device/bridge/device/device.go:317 +0x183
github.com/plgd-dev/device/v2/bridge/device/cloud.(*Manager).serveCOAP(0xc000280000, {0x935bc0, 0xc00113c088}, 0xc00011d840)
	/go/src/github.com/plgd-dev/device/bridge/device/cloud/manager.go:464 +0x1eb
github.com/plgd-dev/go-coap/v3/mux.HandlerFunc.ServeCOAP(0xc0001865a0?, {0x935bc0?, 0xc00113c088?}, 0xc0001865a0?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/router.go:36 +0x29
github.com/plgd-dev/device/v2/bridge/device/cloud.(*Manager).dial.CreateLoggingMiddleware.func8.1({0x935bc0, 0xc00113c088}, 0xc00011d840)
	/go/src/github.com/plgd-dev/device/bridge/net/network.go:188 +0x4d
github.com/plgd-dev/go-coap/v3/mux.HandlerFunc.ServeCOAP(0x9326e0?, {0x935bc0?, 0xc00113c088?}, 0xc0001864e0?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/router.go:36 +0x29
github.com/plgd-dev/go-coap/v3/mux.(*Router).ServeCOAP(0xc0000d0100, {0x935bc0, 0xc00113c088}, 0xc00011d840)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/router.go:236 +0x202
github.com/plgd-dev/go-coap/v3/options.(*MuxHandlerOpt).TCPClientApply.MuxHandlerOpt.TCPClientApply.ToHandler[...].func1(0xc000216800)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/muxResponseWriter.go:18 +0xde
github.com/plgd-dev/go-coap/v3/net/observation.(*Handler[...]).Handle(0x93b600, 0xc001139338, 0xc000216800)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/net/observation/handler.go:44 +0x133
github.com/plgd-dev/go-coap/v3/tcp/client.(*Conn).handle(0xc00019c180, 0xc001139338, 0xc000216800)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:362 +0x196
github.com/plgd-dev/go-coap/v3/tcp/client.(*Conn).ProcessReceivedMessageWithHandler(0xc00019c180, 0xc000216800, 0xc00011d830)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:329 +0x22e
github.com/plgd-dev/go-coap/v3/tcp/client.processReceivedMessage(0x4066a0?, 0xc00009a2a0?, 0xc000043730?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:146 +0x1c
github.com/plgd-dev/go-coap/v3/tcp/client.(*Conn).ProcessReceivedMessage(0xc00019c180, 0xc000216800)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:150 +0x62
github.com/plgd-dev/go-coap/v3/net/client.(*ReceivedMessageReader[...]).loop(0x934d40, 0xc000122540, 0xc0013e63d8)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/net/client/receivedMessageReader.go:66 +0x110
created by github.com/plgd-dev/go-coap/v3/net/client.NewReceivedMessageReader[...] in goroutine 36
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/net/client/receivedMessageReader.go:41 +0x192

goroutine 289 [runnable]:
sync.(*Map).Load(0xbd37e0, {0x86ff00, 0x7c8c40})
	/usr/local/go/src/sync/map.go:122 +0x4f
encoding/json.typeEncoder({0x93c988, 0x7c8c40})
	/usr/local/go/src/encoding/json/encode.go:343 +0x45
encoding/json.valueEncoder({0x7c8c40?, 0xc000239480?, 0x30?})
	/usr/local/go/src/encoding/json/encode.go:339 +0x45
encoding/json.(*encodeState).reflectValue(0xc0010889c0, {0x7c8c40?, 0xc000239480?, 0x40e785?}, {0x38?, 0x0?})
	/usr/local/go/src/encoding/json/encode.go:321 +0x45
encoding/json.(*encodeState).marshal(0xc001089f80?, {0x7c8c40?, 0xc000239480?}, {0x70?, 0x8d?})
	/usr/local/go/src/encoding/json/encode.go:297 +0xc5
encoding/json.Marshal({0x7c8c40, 0xc000239480})
	/usr/local/go/src/encoding/json/encode.go:163 +0xd0
github.com/web-of-things-open-source/thingdescription-go/thingDescription.marshalUnion(0xc001078e80?, 0x40bb7f?, 0x802e20?, 0x7fed20?, 0x0?, {0x7d3e80?, 0xc37ae0?}, 0x60?, {0x0, 0x0}, ...)
	/go/pkg/mod/github.com/web-of-things-open-source/thingdescription-go@v0.0.0-20240513190706-79b5f39190eb/thingDescription/JSONSchemaSupport.go:114 +0xd0
github.com/web-of-things-open-source/thingdescription-go/thingDescription.(*ThingContext).MarshalJSON(0xc0001d5660)
	/go/pkg/mod/github.com/web-of-things-open-source/thingdescription-go@v0.0.0-20240513190706-79b5f39190eb/thingDescription/ThingContext.go:21 +0x8d
github.com/go-json-experiment/json.makeMethodArshaler.func2(0xc0010de420, {{0x819120?, 0xc0001d5660?, 0x0?}}, 0xc001078ff8?)
	/go/pkg/mod/github.com/go-json-experiment/json@v0.0.0-20240418180308-af2d5061e6c2/arshal_methods.go:134 +0xc2
github.com/go-json-experiment/json.makeDefaultArshaler.makePointerArshaler.func16(0xc0010de420, {{0x7fed20?, 0xc000284b60?, 0xc0001661c8?}}, 0xc0010de4e8)
	/go/pkg/mod/github.com/go-json-experiment/json@v0.0.0-20240418180308-af2d5061e6c2/arshal_default.go:1474 +0x2cd
github.com/go-json-experiment/json.makeDefaultArshaler.makeStructArshaler.func9(0xc0010de420, {{0x8248e0?, 0xc0011061e0?, 0xc001079228?}}, 0xc0010de4e8)
	/go/pkg/mod/github.com/go-json-experiment/json@v0.0.0-20240418180308-af2d5061e6c2/arshal_default.go:1014 +0xaf7
github.com/go-json-experiment/json.marshalEncode(0xc0010de420, {0x7d3480?, 0xc0011061e0?}, 0xc0010de4e8)
	/go/pkg/mod/github.com/go-json-experiment/json@v0.0.0-20240418180308-af2d5061e6c2/arshal.go:218 +0x1fd
github.com/go-json-experiment/json.Marshal({0x7d3480, 0xc0011061e0}, {0x0?, 0xa341ecf171314602?, 0xde8022dfa0cde2bc?})
	/go/pkg/mod/github.com/go-json-experiment/json@v0.0.0-20240418180308-af2d5061e6c2/arshal.go:168 +0xcd
github.com/web-of-things-open-source/thingdescription-go/thingDescription.ThingDescription.MarshalJSON({0xc0001d5660, 0xc0001d5680, 0x0, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, 0x0}, ...}, ...})
	/go/pkg/mod/github.com/web-of-things-open-source/thingdescription-go@v0.0.0-20240513190706-79b5f39190eb/thingDescription/ThingDescription.go:50 +0x129
github.com/plgd-dev/device/v2/bridge/resources/thingDescription.(*Resource).createMessage(0x9358e8?, 0xc001106150, 0x0?)
	/go/src/github.com/plgd-dev/device/bridge/resources/thingDescription/resource.go:59 +0x98
github.com/plgd-dev/device/v2/bridge/resources/thingDescription.(*Resource).Get(0xc000199680, 0xc001106150)
	/go/src/github.com/plgd-dev/device/bridge/resources/thingDescription/resource.go:92 +0x5a
github.com/plgd-dev/device/v2/bridge/resources.(*Resource).observerHandler(0xc00027ea10, 0xc001106150, 0xf8?)
	/go/src/github.com/plgd-dev/device/bridge/resources/resource.go:286 +0x138
github.com/plgd-dev/device/v2/bridge/resources.(*Resource).HandleRequest(0xc00027ea10, 0xc001106150)
	/go/src/github.com/plgd-dev/device/bridge/resources/resource.go:317 +0x8d
github.com/plgd-dev/device/v2/bridge/device.(*Device).HandleRequest(0xc000284000, 0xc001106150)
	/go/src/github.com/plgd-dev/device/bridge/device/device.go:317 +0x183
github.com/plgd-dev/device/v2/bridge/device/cloud.(*Manager).serveCOAP(0xc0002803c0, {0x935bc0, 0xc0000a00b8}, 0xc0013436c0)
	/go/src/github.com/plgd-dev/device/bridge/device/cloud/manager.go:464 +0x1eb
github.com/plgd-dev/go-coap/v3/mux.HandlerFunc.ServeCOAP(0xc001106120?, {0x935bc0?, 0xc0000a00b8?}, 0xc001106120?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/router.go:36 +0x29
github.com/plgd-dev/device/v2/bridge/device/cloud.(*Manager).dial.CreateLoggingMiddleware.func8.1({0x935bc0, 0xc0000a00b8}, 0xc0013436c0)
	/go/src/github.com/plgd-dev/device/bridge/net/network.go:188 +0x4d
github.com/plgd-dev/go-coap/v3/mux.HandlerFunc.ServeCOAP(0x9326e0?, {0x935bc0?, 0xc0000a00b8?}, 0xc0011060f0?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/router.go:36 +0x29
github.com/plgd-dev/go-coap/v3/mux.(*Router).ServeCOAP(0xc001088040, {0x935bc0, 0xc0000a00b8}, 0xc0013436c0)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/router.go:236 +0x202
github.com/plgd-dev/go-coap/v3/options.(*MuxHandlerOpt).TCPClientApply.MuxHandlerOpt.TCPClientApply.ToHandler[...].func1(0xc000216100)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/mux/muxResponseWriter.go:18 +0xde
github.com/plgd-dev/go-coap/v3/net/observation.(*Handler[...]).Handle(0x93b600, 0xc0014c4f48, 0xc000216100)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/net/observation/handler.go:44 +0x133
github.com/plgd-dev/go-coap/v3/tcp/client.(*Conn).handle(0xc00126c720, 0xc0014c4f48, 0xc000216100)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:362 +0x196
github.com/plgd-dev/go-coap/v3/tcp/client.(*Conn).ProcessReceivedMessageWithHandler(0xc00126c720, 0xc000216100, 0xc0013436b0)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:329 +0x22e
github.com/plgd-dev/go-coap/v3/tcp/client.processReceivedMessage(0x4066a0?, 0xc001092120?, 0xc000042f30?)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:146 +0x1c
github.com/plgd-dev/go-coap/v3/tcp/client.(*Conn).ProcessReceivedMessage(0xc00126c720, 0xc000216100)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/tcp/client/conn.go:150 +0x62
github.com/plgd-dev/go-coap/v3/net/client.(*ReceivedMessageReader[...]).loop(0x934d40, 0xc0010245a0, 0xc00019a978)
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/net/client/receivedMessageReader.go:66 +0x110
created by github.com/plgd-dev/go-coap/v3/net/client.NewReceivedMessageReader[...] in goroutine 40
	/go/pkg/mod/github.com/plgd-dev/go-coap/v3@v3.3.4/net/client/receivedMessageReader.go:41 +0x192

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of Thing Description (TD) generation to ensure it always completes, even if an error occurs during the initial attempt.

Copy link

coderabbitai bot commented May 22, 2024

Walkthrough

The update to the getTDOpts function in main.go ensures that the Thing Description (TD) is consistently generated, even in the presence of errors during the initial attempt. This improvement enhances concurrency handling by guaranteeing the creation of a TD regardless of any initial errors.

Changes

File Change Summary
cmd/bridge-device/main.go Modified getTDOpts function to ensure the creation of Thing Description (TD) always, handling errors by generating a TD even if an error occurs during the initial attempt.

🐇 In code we trust, we make it right,
Ensuring TDs even in the night.
Concurrency strong, errors no fright,
Bridge-device shines, ever so bright.
🌟✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between 13e2819 and b8eb672.
Files selected for processing (1)
  • cmd/bridge-device/main.go (1 hunks)

cmd/bridge-device/main.go Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented May 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.81%. Comparing base (6021006) to head (9f492ed).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #470      +/-   ##
==========================================
- Coverage   77.90%   76.81%   -1.10%     
==========================================
  Files         118      121       +3     
  Lines        7206     7225      +19     
==========================================
- Hits         5614     5550      -64     
- Misses       1029     1087      +58     
- Partials      563      588      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jkralik jkralik force-pushed the jkralik/fix/fatal-error-concurrent-map-writes branch from b8eb672 to 9f492ed Compare May 22, 2024 11:49
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between b8eb672 and 9f492ed.
Files selected for processing (1)
  • cmd/bridge-device/main.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
  • cmd/bridge-device/main.go

Copy link

sonarcloud bot commented May 22, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@jkralik jkralik merged commit c91027b into main May 22, 2024
14 checks passed
@jkralik jkralik deleted the jkralik/fix/fatal-error-concurrent-map-writes branch May 22, 2024 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants