Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify message format to publish on the bus #4

Closed
3 tasks done
simonswine opened this issue Jul 23, 2020 · 4 comments · Fixed by #10
Closed
3 tasks done

Verify message format to publish on the bus #4

simonswine opened this issue Jul 23, 2020 · 4 comments · Fixed by #10

Comments

@simonswine
Copy link
Collaborator

simonswine commented Jul 23, 2020

The message format (cf. https://github.com/grafana/prometheus-pulsar-remote-write/blob/7d5901bb9d0ac60313a50b85a309949ccbeb2a10/pulsar/serializers_test.go) how metrics are represented is based on the provided implementation from https://github.com/liangyuanpeng/prometheus-pulsar-adapter.

I think we should review this going forward and think about those conditions:

  • We need to make sure we have a lossless marshal and unmarshaling of the payload (esp. for phase 2 when we trying to read back the messages from the bus)
  • Potentially the customer has some custom requirements or other legacy format that we need to match
  • We might need to store more information (e.g) tenant ID
  • We should also consider using the properties fields and/or the EventTime of pulsar Messages (not for now)
@simonswine
Copy link
Collaborator Author

@jtlisi I think this is the next step for this adapter. Let me know what you think and if we have any idea in which format the customer expect the messages to be sent

@jtlisi
Copy link

jtlisi commented Jul 27, 2020

I agree, we are going to need to store the tenant ID. The ideal format for these messages will be JSON initially, if another format is requested we can add support for it later.

@simonswine
Copy link
Collaborator Author

We should remove AVRO from the CLI and publish the tenant ID in JSON format

@simonswine simonswine added this to the v0.1 milestone Jul 31, 2020
@simonswine
Copy link
Collaborator Author

An important point to remember is that the tenant ID should be acquired through basic auth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants