Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No support for streams timestamp offset? #722

Closed
snird opened this issue Jun 11, 2023 · 8 comments
Closed

No support for streams timestamp offset? #722

snird opened this issue Jun 11, 2023 · 8 comments

Comments

@snird
Copy link

snird commented Jun 11, 2023

The example for streams: 4993dee

says we can specify a timestamp.
But by specifying a number for the x-stream-offset header argument, RabbitMQ uses it as the offset number, not timestamp.

I looked at RabbitMQ source code, and their test suite for the case makes it clear:
https://github.com/rabbitmq/rabbitmq-server/blob/ddb9fbd12cc8e7c38740d48a226f770d4878dcf9/deps/rabbit/test/rabbit_stream_queue_SUITE.erl#L954

For timestamp, the argument type must be "timestamp". If it is "long", as it seems to be the case with this lib, it uses it as the offset directly, not as timestamp.

Is there a way to make timestamp offsets work?

@snird
Copy link
Author

snird commented Jun 11, 2023

I have tried to use the coded of this lib: https://github.com/amqp-node/amqplib/blob/4993dee433665e9e6fe7729f3eaeacca06b46cb7/lib/codec.js

but so far I was not succesful.
I guess that I have to use it, to encode a timestamp correctly.

Can someone shed some light on how to do that?

@snird
Copy link
Author

snird commented Jun 12, 2023

I'm now using https://github.com/coders51/rabbitmq-stream-js-client instead.
It required some work as it is still experimental, but mostly merging locally the open PRs was enough.
I only consume through the rabbitmq-stream-js-client lib, to publish and create I still do through amqplib.

Just letting people here know in case they try to use streams.

Maybe worth noting in the "streams example" in this repo that timestamp offsets don't work currently.

@cressie176
Copy link
Collaborator

Hi @snird,

Thank you. I did have a look over the weekend but couldn't get the streams example working either. Do you have a public repo with the working version of rabbitmq-stream-js-client? I'd like to see what the x-stream-offset timestamp looks like with Wireshark and compare with amqplib

Thanks

Steve

@snird
Copy link
Author

snird commented Jun 12, 2023

I don't have it public, as it is still on testing phase on my side.

Anyway, that won't help your case: the https://github.com/coders51/rabbitmq-stream-js-client lib uses the binary format of streams in RabbitMQ. That's not amqp 0.9.1 at all, so there are no possible comparisons.

@cressie176
Copy link
Collaborator

OK, thanks anyway

@cressie176
Copy link
Collaborator

You can force a timestamp using the following syntax

x-stream-offset’: { ‘!’: ‘timestamp’, value: 1686519750 }

Where 1686519750 is the number of seconds (as opposed to miliseconds) since 00:00:00 UTC, 1970-01-01

@cressie176
Copy link
Collaborator

I'll leave this issue open until I've added something to the streams example & documentation

@cressie176
Copy link
Collaborator

Done. Thanks for opening this issue @snird, it was interesting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants