Skip to content

2.29.0.0-b88

@jaki jaki tagged this 22 Oct 22:50
Summary:
Index backfill code has messy type conversions involving size_t,
uint64_t, int64_t.  The main long-lasting value is the number of table
rows processed by backfill.  It is not possible to avoid all conversions
because the source and destination types are different and there are
some stored metadata items, namely BackfillJobPB.num_rows_processed and
IndexInfoPB.num_rows_processed_by_backfill_job in the middle, whose
types cannot be changed easily.  Settle on the following:

- Use uint64_t for most of the lifetime since this is the type used by
  the stored metadata items.
- Switch to int64_t only at the end (pg_stat_get_progress_info) as PG
  expects int8 format.

As for using uint64_t, thankfully, it is already mostly consistent in
the upper levels.  It only starts to break down in the tserver to
postgres to tserver level at the deep end.  Since this is all concerned
with a single node, we don't have to worry about rolling upgrade issues.

As for switching to int64_t, that already existed, but the cast was done
transparently and is not safe.  Add a check to avoid overflow.
Practically, this should never have been an issue because the number is
too high to occur in real world cases.

Of particular note:

- YCQL backfill code is also changed to use uint64_t in some areas
  because code is shared with YSQL backfill.
- The limit in the spec was an int64 with default -1, but said default
  is never used since the limit is always set.  So remove the default
  and make the field uint64.

**Upgrade/Rollback safety:**
As mentioned above, proto changes are only related to a single node.

Jira: DB-18795

Test Plan:
Jenkins

Close: #29038

Reviewers: gkukreja

Reviewed By: gkukreja

Subscribers: ybase, yql

Differential Revision: https://phorge.dev.yugabyte.com/D47638
Assets 2
Loading