Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistent, documented resolved-timestamp format for EXPERIMENTAL CHANGEFEED #42603

Open
cretz opened this issue Nov 20, 2019 · 3 comments
Open

Consistent, documented resolved-timestamp format for EXPERIMENTAL CHANGEFEED #42603

cretz opened this issue Nov 20, 2019 · 3 comments
Labels
Projects

Comments

@cretz
Copy link

@cretz cretz commented Nov 20, 2019

Per this forum post, I have seen (in 19.1 and I believe 19.2) resolved timestamps come across as both { "__crdb__": { "resolved": "someval" }} and { "resolved": "someval" }. Please strictly define and document the expected resolved timestamp format.

@ricardocrdb ricardocrdb added this to To do in Support via automation Nov 20, 2019
@ricardocrdb

This comment has been minimized.

Copy link
Collaborator

@ricardocrdb ricardocrdb commented Nov 22, 2019

Hey @cretz

As @danhhz had responded to in that forum post after these GH issues were created, I will paste his response here:
In 19.1, we changed the default format for newly created changefeeds from format=row to format=wrapped. One of these feeds (the one without crdb) was created with a pre 19.1 cluster and still has format=row. If you want them to be the same, you can stop one of them and replace it with a new changefeed that explicitly specifies the other format.

Let me know if you have any other questions.

@ricardocrdb ricardocrdb moved this from To do to Triaging in Support Nov 22, 2019
@cretz

This comment has been minimized.

Copy link
Author

@cretz cretz commented Nov 22, 2019

@ricardocrdb - I request that this be documented please. Receiving "resolved" timestamps is important, so it's only fair that people know what format it comes in. Also, in the docs I don't see anything about setting format to row or wrapped or anything of the sort, only json or experimental_avro.

@danhhz

This comment has been minimized.

Copy link
Contributor

@danhhz danhhz commented Nov 22, 2019

Hey @cretz. My apologies, I misspoke on the forum. It's envelope=row vs envelope=wrapped. Not format, which as you point out switches between avro and json.

@lnhsingh Looks like the EXPERIMENTAL CHANGEFEED FOR page documents row and key_only for envelope= and CREATE CHANGEFEED documents wrapped and key_only. All three options work for either, so this seems out of date.

Additionally, we called out in release notes that the default value for envelope changed, but given how long running changefeeds are, I can see how that would not be enough. I'm not sure how to best communicate this in our docs, but it seems worthwhile to add it somewhere.

Also cc @rolandcrosby

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Support
  
Triaging
3 participants
You can’t perform that action at this time.