Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pubsub/natspubsub: NATS v2.2.0+ native message headers and message encoding #3282

Merged
merged 8 commits into from
Jul 21, 2023

Conversation

kucjac
Copy link
Contributor

@kucjac kucjac commented Jul 21, 2023

This PR implements the native NATS v2.2.0+ message headers as metadata.

This PR doesn't break the API of the github.com/google/go-cloud/pubsub/natspubsub package. It provides additional functionality that could be chosen by the user of the library.

For servers that support it (NATS Server 2.2.0 or later), messages could
be encoded using native NATS message headers, and native message content.
This provides full support for non-Go clients. Versions prior to 2.2.0 uses gob.Encoder to encode the message metadata and content, which limits the subscribers only to Go clients.
To use this feature, set the query parameter "natsv2" in the URL.
The value needs to either be empty value or boolean parsable. For example:

  • nats://mysubject?natsv2
  • nats://mysubject?natsv2=true

This feature could also be enabled by setting the URLOpener.UseV2 field manually.

This implementation automatically determines if the server is capable of using v2.2.0+ headers.
If the server does not support it, any attempt to use it results in an error.

Using native NATS message headers and content is more efficient than using
gob.Encoder, and allows non-Go clients to subscribe to the topic and
receive messages.
The benchmarks provided in the test files shows that using v2.2.0+ API is about three times faster on both Send and Receive operations than the original implementation:

goos: linux
goarch: amd64
pkg: gocloud.dev/pubsub/natspubsub
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkNatsQueuePubSub
BenchmarkNatsQueuePubSub/V1
BenchmarkNatsQueuePubSub/V1/BenchmarkReceive
BenchmarkNatsQueuePubSub/V1/BenchmarkReceive-16	1	1368954367 ns/op 7304.85 MB/s
BenchmarkNatsQueuePubSub/V1/BenchmarkSend
BenchmarkNatsQueuePubSub/V1/BenchmarkSend-16	12	102359676 ns/op	97694.72 MB/s
BenchmarkNatsQueuePubSub/V2
BenchmarkNatsQueuePubSub/V2/BenchmarkReceive
BenchmarkNatsQueuePubSub/V2/BenchmarkReceive-16	3	439734654 ns/op	22740.99 MB/s
BenchmarkNatsQueuePubSub/V2/BenchmarkSend
BenchmarkNatsQueuePubSub/V2/BenchmarkSend-16	28	44177761 ns/op	226358.23 MB/s
PASS

where the MB/s is the actual number of messages send/received per second.

@codecov
Copy link

codecov bot commented Jul 21, 2023

Codecov Report

Merging #3282 (315b8a0) into master (8f12933) will decrease coverage by 0.09%.
The diff coverage is 69.75%.

@@            Coverage Diff             @@
##           master    #3282      +/-   ##
==========================================
- Coverage   77.47%   77.39%   -0.09%     
==========================================
  Files         104      104              
  Lines       13721    13853     +132     
==========================================
+ Hits        10631    10721      +90     
- Misses       2353     2374      +21     
- Partials      737      758      +21     
Impacted Files Coverage Δ
pubsub/natspubsub/nats.go 72.58% <69.75%> (-3.08%) ⬇️

pubsub/natspubsub/nats.go Outdated Show resolved Hide resolved
pubsub/natspubsub/nats.go Outdated Show resolved Hide resolved
pubsub/natspubsub/nats.go Outdated Show resolved Hide resolved
pubsub/natspubsub/nats.go Outdated Show resolved Hide resolved
@kucjac
Copy link
Contributor Author

kucjac commented Jul 21, 2023

Thank you, @vangent for your review. I've already applied requested changes.

@vangent
Copy link
Contributor

vangent commented Jul 21, 2023

Thanks for the contribution!

@vangent vangent merged commit 93f8432 into google:master Jul 21, 2023
ybourgery pushed a commit to Simprints/go-cloud that referenced this pull request Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants