arc: add output plugin for Arc columnar database#4265
Conversation
|
Commits Review
|
6d99dc9 to
f638a2d
Compare
| batch := service.MessageBatch{ | ||
| service.NewMessage([]byte(`{"sensor":"temp-1","value":22.5}`)), | ||
| service.NewMessage([]byte(`{"sensor":"temp-2","value":23.1}`)), | ||
| } | ||
|
|
||
| require.NoError(t, out.WriteBatch(t.Context(), batch)) | ||
|
|
There was a problem hiding this comment.
Bug: rowCount and err in the require.Eventually format args are evaluated at call time — before the polling loop runs — so they will always be 0 and nil (their zero values). On failure the message will read "expected N rows, got 0 (last err: <nil>)" regardless of what actually happened, making CI failures hard to debug.
Fix: format the message inside the condition closure instead. For example, capture a lastMsg string variable and update it each iteration, then use require.Eventuallyf or move the assertion outside the Eventually and check the final values after the loop.
|
Commits Review
|
Co-Authored-By: Michał Matczuk <michal.matczuk@redpanda.com>
f638a2d to
4df201e
Compare
|
Commits Review LGTM |
|
Why doing this? Hard to track contributors, doesn't feel like a good practice. Can you clarify? |
|
@xe-nvdk to streamline things on your end like failing CI and to be able to include it in a release. Cheers. |
| } | ||
|
|
||
| rec.columns["time"] = append(rec.columns["time"], ts) | ||
| for k, v := range dataMap { |
There was a problem hiding this comment.
Should we be concerned with map's random ordering here?
| rec.columns["time"] = append(rec.columns["time"], ts) | ||
| for k, v := range dataMap { | ||
| if k == o.timestampField || k == "time" { | ||
| continue |
There was a problem hiding this comment.
Should we document/log that any message field named "time" is dropped silently?
| default: | ||
| return fmt.Errorf("unsupported format: %s", o.format) | ||
| } | ||
| if err != nil { |
There was a problem hiding this comment.
Given we return on err here, would it not be better returning earlier (within the case)?
| return nil | ||
| } | ||
|
|
||
| var payload any |
There was a problem hiding this comment.
suggestion:
var (
payload any
err error
)|
|
||
| data, err := msg.AsStructuredMut() | ||
| if err != nil { | ||
| return nil, fmt.Errorf("message %d: %w", i, err) |
There was a problem hiding this comment.
Not sure how helpful this error message prefix is (message %d), Is there a clearer prefix we could use?
|
None of these comments are blockers and can be addressed post merge. |
No description provided.