Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spanconfigsqlwatcher: add a system.protected_ts_records decoder #74913

Merged
merged 1 commit into from
Jan 19, 2022

Conversation

adityamaru
Copy link
Contributor

This change adds a decoder to decode rows from the
system.protected_ts_records table. This will be used by the
rangefeed we setup in the SQLWatcher, to decode rangefeed updates
and subsequently feed them into the SQLTranslator. The target column
of the system table is the only column of interest since it will
determine whether we ask the translator to generate SystemSpanConfigs
or regular SpanConfigs. More information about this will be added
in follow up PRs that use this decoder.

This change also instructs gogoproto to generate an equality method
for roachpb.TenantID, to allow for equality checks in the decoder
test.

Informs: #73727

Release note: None

@adityamaru adityamaru requested a review from a team as a code owner January 17, 2022 17:24
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@adityamaru
Copy link
Contributor Author

@dt adding you for the small change to roachpb.TenantID

@dt
Copy link
Member

dt commented Jan 17, 2022

for the small change to roachpb.TenantID

I'll defer to @tbg or @RaduBerinde if we want the proto message to be comparable (I'd guess yes -- the go struct was already comparable -- so I don't see why not, but we treat TenantID with care so let's check)

Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, I don't see why not.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arulajmani and @dt)

@dt dt removed their request for review January 18, 2022 03:12
@tbg
Copy link
Member

tbg commented Jan 18, 2022

That seems ok.

s0 := tc.Server(0)
ptp := s0.DistSQLServer().(*distsql.ServerImpl).ServerConfig.ProtectedTimestampProvider
jr := s0.JobRegistry().(*jobs.Registry)
k := keys.SystemSQLCodec.TablePrefix(keys.ProtectedTimestampsRecordsTableID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of writing to the real table in this unit test, it'd be better isolated if using a scratch table instead:

const dummyTableName = "dummy_span_configurations"
tdb := sqlutils.MakeSQLRunner(tc.ServerConn(0))
tdb.Exec(t, fmt.Sprintf("CREATE TABLE %s (LIKE system.span_configurations INCLUDING ALL)", dummyTableName))

Given this test should only care about the decoding routines, I would suggest writing to the scratch table in this way directly, instead of depending on jobs or the protected ts subsystem to write to the real table (and also needing to release the record). Also, since the column is nullable, worth having a nil check (an empty proto should be returned -- something we'll need to explicitly handle).

Copy link
Contributor Author

@adityamaru adityamaru Jan 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm doesn't it make sense to test this with the jobsprotectedts.MakeRecord, Protect, and Release methods that are actually going to be used to interact with this table by all PTS consumers? We use this pattern in many other PTS tests as well. In theory, target is a free-form bytes column and the decoder+test could be doing something different from what the actual system writes to the column, so it seems a little scary to decouple it from how a pts record will actually be written?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'm not sure, but I see your point. I was coming at it to minimize the test/make it fast enough to also then suggest writing a benchmark for it ("TestProtectedTimestampDecoder only cares about decoding"). Minimizing the test also prevents it from failing when another subsystem is buggy (Protect/Release); the nice thing about talking through this table is that these sub-systems are decoupled, I think it's fine/sane to have them decoupled in test code as well. I'll defer to you and not hold up the PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minimizing the test also prevents it from failing when another subsystem is buggy

Coming from bulkio land where we write almost all tests that rely on the jobs subsystem, this is something I've just taken for granted before heh. The benchmarking is a good point, I'm leaning towards merging this for now since I have CI happy, and cleaning it up during stability if we decide to benchmark some of these pieces.

This change adds a decoder to decode rows from the
`system.protected_ts_records` table. This will be used by the
rangefeed we setup in the SQLWatcher, to decode rangefeed updates
and subsequently feed them into the SQLTranslator. The `target` column
of the system table is the only column of interest since it will
determine whether we ask the translator to generate `SystemSpanConfig`s
or regular `SpanConfigs`. More information about this will be added
in follow up PRs that use this decoder.

This change also instructs gogoproto to generate an equality method
for `roachpb.TenantID`, to allow for equality checks in the decoder
test.

Informs: cockroachdb#73727

Release note: None
@adityamaru
Copy link
Contributor Author

TFTR!

bors r=irfansharif

@craig
Copy link
Contributor

craig bot commented Jan 19, 2022

Build succeeded:

@craig craig bot merged commit b917f1e into cockroachdb:master Jan 19, 2022
@adityamaru adityamaru deleted the pts-row-decoder branch January 19, 2022 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants