Skip to content

Commit

Permalink
Refactor how VACUUM passes around its XID cutoffs.
Browse files Browse the repository at this point in the history
Use a dedicated struct for the XID/MXID cutoffs used by VACUUM, such as
FreezeLimit and OldestXmin.  This state is initialized in vacuum.c, and
then passed around by code from vacuumlazy.c to heapam.c freezing
related routines.  The new convention is that everybody works off of the
same cutoff state, which is passed around via pointers to const.

Also simplify some of the logic for dealing with frozen xmin in
heap_prepare_freeze_tuple: add dedicated "xmin_already_frozen" state to
clearly distinguish xmin XIDs that we're going to freeze from those that
were already frozen from before.  That way the routine's xmin handling
code is symmetrical with the existing xmax handling code.  This is
preparation for an upcoming commit that will add page level freezing.

Also refactor the control flow within FreezeMultiXactId(), while adding
stricter sanity checks.  We now test OldestXmin directly, instead of
using FreezeLimit as an inexact proxy for OldestXmin.  This is further
preparation for the page level freezing work, which will make the
function's caller cede control of page level freezing to the function
where appropriate (where heap_prepare_freeze_tuple sees a tuple that
happens to contain a MultiXactId in its xmax).

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Jeff Davis <pgsql@j-davis.com>
Discussion: https://postgr.es/m/CAH2-WznS9TxXmz2_=SY+SyJyDFbiOftKofM9=aDo68BbXNBUMA@mail.gmail.com
  • Loading branch information
petergeoghegan committed Dec 22, 2022
1 parent e42e312 commit 4ce3afb
Show file tree
Hide file tree
Showing 8 changed files with 431 additions and 458 deletions.
478 changes: 232 additions & 246 deletions src/backend/access/heap/heapam.c

Large diffs are not rendered by default.

196 changes: 81 additions & 115 deletions src/backend/access/heap/vacuumlazy.c

Large diffs are not rendered by default.

9 changes: 3 additions & 6 deletions src/backend/access/transam/multixact.c
Expand Up @@ -2813,14 +2813,11 @@ ReadMultiXactCounts(uint32 *multixacts, MultiXactOffset *members)
* As the fraction of the member space currently in use grows, we become
* more aggressive in clamping this value. That not only causes autovacuum
* to ramp up, but also makes any manual vacuums the user issues more
* aggressive. This happens because vacuum_set_xid_limits() clamps the
* freeze table and the minimum freeze age based on the effective
* aggressive. This happens because vacuum_get_cutoffs() will clamp the
* freeze table and the minimum freeze age cutoffs based on the effective
* autovacuum_multixact_freeze_max_age this function returns. In the worst
* case, we'll claim the freeze_max_age to zero, and every vacuum of any
* table will try to freeze every multixact.
*
* It's possible that these thresholds should be user-tunable, but for now
* we keep it simple.
* table will freeze every multixact.
*/
int
MultiXactMemberFreezeThreshold(void)
Expand Down
25 changes: 12 additions & 13 deletions src/backend/commands/cluster.c
Expand Up @@ -826,10 +826,7 @@ copy_table_data(Oid OIDNewHeap, Oid OIDOldHeap, Oid OIDOldIndex, bool verbose,
TupleDesc oldTupDesc PG_USED_FOR_ASSERTS_ONLY;
TupleDesc newTupDesc PG_USED_FOR_ASSERTS_ONLY;
VacuumParams params;
TransactionId OldestXmin,
FreezeXid;
MultiXactId OldestMxact,
MultiXactCutoff;
struct VacuumCutoffs cutoffs;
bool use_sort;
double num_tuples = 0,
tups_vacuumed = 0,
Expand Down Expand Up @@ -918,23 +915,24 @@ copy_table_data(Oid OIDNewHeap, Oid OIDOldHeap, Oid OIDOldIndex, bool verbose,
* not to be aggressive about this.
*/
memset(&params, 0, sizeof(VacuumParams));
vacuum_set_xid_limits(OldHeap, &params, &OldestXmin, &OldestMxact,
&FreezeXid, &MultiXactCutoff);
vacuum_get_cutoffs(OldHeap, &params, &cutoffs);

/*
* FreezeXid will become the table's new relfrozenxid, and that mustn't go
* backwards, so take the max.
*/
if (TransactionIdIsValid(OldHeap->rd_rel->relfrozenxid) &&
TransactionIdPrecedes(FreezeXid, OldHeap->rd_rel->relfrozenxid))
FreezeXid = OldHeap->rd_rel->relfrozenxid;
TransactionIdPrecedes(cutoffs.FreezeLimit,
OldHeap->rd_rel->relfrozenxid))
cutoffs.FreezeLimit = OldHeap->rd_rel->relfrozenxid;

/*
* MultiXactCutoff, similarly, shouldn't go backwards either.
*/
if (MultiXactIdIsValid(OldHeap->rd_rel->relminmxid) &&
MultiXactIdPrecedes(MultiXactCutoff, OldHeap->rd_rel->relminmxid))
MultiXactCutoff = OldHeap->rd_rel->relminmxid;
MultiXactIdPrecedes(cutoffs.MultiXactCutoff,
OldHeap->rd_rel->relminmxid))
cutoffs.MultiXactCutoff = OldHeap->rd_rel->relminmxid;

/*
* Decide whether to use an indexscan or seqscan-and-optional-sort to scan
Expand Down Expand Up @@ -973,13 +971,14 @@ copy_table_data(Oid OIDNewHeap, Oid OIDOldHeap, Oid OIDOldIndex, bool verbose,
* values (e.g. because the AM doesn't use freezing).
*/
table_relation_copy_for_cluster(OldHeap, NewHeap, OldIndex, use_sort,
OldestXmin, &FreezeXid, &MultiXactCutoff,
cutoffs.OldestXmin, &cutoffs.FreezeLimit,
&cutoffs.MultiXactCutoff,
&num_tuples, &tups_vacuumed,
&tups_recently_dead);

/* return selected values to caller, get set as relfrozenxid/minmxid */
*pFreezeXid = FreezeXid;
*pCutoffMulti = MultiXactCutoff;
*pFreezeXid = cutoffs.FreezeLimit;
*pCutoffMulti = cutoffs.MultiXactCutoff;

/* Reset rd_toastoid just to be tidy --- it shouldn't be looked at again */
NewHeap->rd_toastoid = InvalidOid;
Expand Down
120 changes: 55 additions & 65 deletions src/backend/commands/vacuum.c
Expand Up @@ -907,34 +907,20 @@ get_all_vacuum_rels(int options)
}

/*
* vacuum_set_xid_limits() -- compute OldestXmin and freeze cutoff points
* vacuum_get_cutoffs() -- compute OldestXmin and freeze cutoff points
*
* The target relation and VACUUM parameters are our inputs.
*
* Our output parameters are:
* - OldestXmin is the Xid below which tuples deleted by any xact (that
* committed) should be considered DEAD, not just RECENTLY_DEAD.
* - OldestMxact is the Mxid below which MultiXacts are definitely not
* seen as visible by any running transaction.
* - FreezeLimit is the Xid below which all Xids are definitely frozen or
* removed during aggressive vacuums.
* - MultiXactCutoff is the value below which all MultiXactIds are definitely
* removed from Xmax during aggressive vacuums.
* Output parameters are the cutoffs that VACUUM caller should use.
*
* Return value indicates if vacuumlazy.c caller should make its VACUUM
* operation aggressive. An aggressive VACUUM must advance relfrozenxid up to
* FreezeLimit (at a minimum), and relminmxid up to MultiXactCutoff (at a
* minimum).
*
* OldestXmin and OldestMxact are the most recent values that can ever be
* passed to vac_update_relstats() as frozenxid and minmulti arguments by our
* vacuumlazy.c caller later on. These values should be passed when it turns
* out that VACUUM will leave no unfrozen XIDs/MXIDs behind in the table.
*/
bool
vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
TransactionId *OldestXmin, MultiXactId *OldestMxact,
TransactionId *FreezeLimit, MultiXactId *MultiXactCutoff)
vacuum_get_cutoffs(Relation rel, const VacuumParams *params,
struct VacuumCutoffs *cutoffs)
{
int freeze_min_age,
multixact_freeze_min_age,
Expand All @@ -954,6 +940,10 @@ vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
freeze_table_age = params->freeze_table_age;
multixact_freeze_table_age = params->multixact_freeze_table_age;

/* Set pg_class fields in cutoffs */
cutoffs->relfrozenxid = rel->rd_rel->relfrozenxid;
cutoffs->relminmxid = rel->rd_rel->relminmxid;

/*
* Acquire OldestXmin.
*
Expand All @@ -965,14 +955,14 @@ vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
* that only one vacuum process can be working on a particular table at
* any time, and that each vacuum is always an independent transaction.
*/
*OldestXmin = GetOldestNonRemovableTransactionId(rel);
cutoffs->OldestXmin = GetOldestNonRemovableTransactionId(rel);

if (OldSnapshotThresholdActive())
{
TransactionId limit_xmin;
TimestampTz limit_ts;

if (TransactionIdLimitedForOldSnapshots(*OldestXmin, rel,
if (TransactionIdLimitedForOldSnapshots(cutoffs->OldestXmin, rel,
&limit_xmin, &limit_ts))
{
/*
Expand All @@ -982,20 +972,48 @@ vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
* frequency), but would still be a significant improvement.
*/
SetOldSnapshotThresholdTimestamp(limit_ts, limit_xmin);
*OldestXmin = limit_xmin;
cutoffs->OldestXmin = limit_xmin;
}
}

Assert(TransactionIdIsNormal(*OldestXmin));
Assert(TransactionIdIsNormal(cutoffs->OldestXmin));

/* Acquire OldestMxact */
*OldestMxact = GetOldestMultiXactId();
Assert(MultiXactIdIsValid(*OldestMxact));
cutoffs->OldestMxact = GetOldestMultiXactId();
Assert(MultiXactIdIsValid(cutoffs->OldestMxact));

/* Acquire next XID/next MXID values used to apply age-based settings */
nextXID = ReadNextTransactionId();
nextMXID = ReadNextMultiXactId();

/*
* Also compute the multixact age for which freezing is urgent. This is
* normally autovacuum_multixact_freeze_max_age, but may be less if we are
* short of multixact member space.
*/
effective_multixact_freeze_max_age = MultiXactMemberFreezeThreshold();

/*
* Almost ready to set freeze output parameters; check if OldestXmin or
* OldestMxact are held back to an unsafe degree before we start on that
*/
safeOldestXmin = nextXID - autovacuum_freeze_max_age;
if (!TransactionIdIsNormal(safeOldestXmin))
safeOldestXmin = FirstNormalTransactionId;
safeOldestMxact = nextMXID - effective_multixact_freeze_max_age;
if (safeOldestMxact < FirstMultiXactId)
safeOldestMxact = FirstMultiXactId;
if (TransactionIdPrecedes(cutoffs->OldestXmin, safeOldestXmin))
ereport(WARNING,
(errmsg("cutoff for removing and freezing tuples is far in the past"),
errhint("Close open transactions soon to avoid wraparound problems.\n"
"You might also need to commit or roll back old prepared transactions, or drop stale replication slots.")));
if (MultiXactIdPrecedes(cutoffs->OldestMxact, safeOldestMxact))
ereport(WARNING,
(errmsg("cutoff for freezing multixacts is far in the past"),
errhint("Close open transactions soon to avoid wraparound problems.\n"
"You might also need to commit or roll back old prepared transactions, or drop stale replication slots.")));

/*
* Determine the minimum freeze age to use: as specified by the caller, or
* vacuum_freeze_min_age, but in any case not more than half
Expand All @@ -1008,19 +1026,12 @@ vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
Assert(freeze_min_age >= 0);

/* Compute FreezeLimit, being careful to generate a normal XID */
*FreezeLimit = nextXID - freeze_min_age;
if (!TransactionIdIsNormal(*FreezeLimit))
*FreezeLimit = FirstNormalTransactionId;
cutoffs->FreezeLimit = nextXID - freeze_min_age;
if (!TransactionIdIsNormal(cutoffs->FreezeLimit))
cutoffs->FreezeLimit = FirstNormalTransactionId;
/* FreezeLimit must always be <= OldestXmin */
if (TransactionIdPrecedes(*OldestXmin, *FreezeLimit))
*FreezeLimit = *OldestXmin;

/*
* Compute the multixact age for which freezing is urgent. This is
* normally autovacuum_multixact_freeze_max_age, but may be less if we are
* short of multixact member space.
*/
effective_multixact_freeze_max_age = MultiXactMemberFreezeThreshold();
if (TransactionIdPrecedes(cutoffs->OldestXmin, cutoffs->FreezeLimit))
cutoffs->FreezeLimit = cutoffs->OldestXmin;

/*
* Determine the minimum multixact freeze age to use: as specified by
Expand All @@ -1035,33 +1046,12 @@ vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
Assert(multixact_freeze_min_age >= 0);

/* Compute MultiXactCutoff, being careful to generate a valid value */
*MultiXactCutoff = nextMXID - multixact_freeze_min_age;
if (*MultiXactCutoff < FirstMultiXactId)
*MultiXactCutoff = FirstMultiXactId;
cutoffs->MultiXactCutoff = nextMXID - multixact_freeze_min_age;
if (cutoffs->MultiXactCutoff < FirstMultiXactId)
cutoffs->MultiXactCutoff = FirstMultiXactId;
/* MultiXactCutoff must always be <= OldestMxact */
if (MultiXactIdPrecedes(*OldestMxact, *MultiXactCutoff))
*MultiXactCutoff = *OldestMxact;

/*
* Done setting output parameters; check if OldestXmin or OldestMxact are
* held back to an unsafe degree in passing
*/
safeOldestXmin = nextXID - autovacuum_freeze_max_age;
if (!TransactionIdIsNormal(safeOldestXmin))
safeOldestXmin = FirstNormalTransactionId;
safeOldestMxact = nextMXID - effective_multixact_freeze_max_age;
if (safeOldestMxact < FirstMultiXactId)
safeOldestMxact = FirstMultiXactId;
if (TransactionIdPrecedes(*OldestXmin, safeOldestXmin))
ereport(WARNING,
(errmsg("cutoff for removing and freezing tuples is far in the past"),
errhint("Close open transactions soon to avoid wraparound problems.\n"
"You might also need to commit or roll back old prepared transactions, or drop stale replication slots.")));
if (MultiXactIdPrecedes(*OldestMxact, safeOldestMxact))
ereport(WARNING,
(errmsg("cutoff for freezing multixacts is far in the past"),
errhint("Close open transactions soon to avoid wraparound problems.\n"
"You might also need to commit or roll back old prepared transactions, or drop stale replication slots.")));
if (MultiXactIdPrecedes(cutoffs->OldestMxact, cutoffs->MultiXactCutoff))
cutoffs->MultiXactCutoff = cutoffs->OldestMxact;

/*
* Finally, figure out if caller needs to do an aggressive VACUUM or not.
Expand Down Expand Up @@ -1113,13 +1103,13 @@ vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
* mechanism to determine if its table's relfrozenxid and relminmxid are now
* dangerously far in the past.
*
* Input parameters are the target relation's relfrozenxid and relminmxid.
*
* When we return true, VACUUM caller triggers the failsafe.
*/
bool
vacuum_xid_failsafe_check(TransactionId relfrozenxid, MultiXactId relminmxid)
vacuum_xid_failsafe_check(const struct VacuumCutoffs *cutoffs)
{
TransactionId relfrozenxid = cutoffs->relfrozenxid;
MultiXactId relminmxid = cutoffs->relminmxid;
TransactionId xid_skip_limit;
MultiXactId multi_skip_limit;
int skip_index_vacuum;
Expand Down
10 changes: 5 additions & 5 deletions src/include/access/heapam.h
Expand Up @@ -38,6 +38,7 @@

typedef struct BulkInsertStateData *BulkInsertState;
struct TupleTableSlot;
struct VacuumCutoffs;

#define MaxLockTupleMode LockTupleExclusive

Expand Down Expand Up @@ -178,8 +179,7 @@ extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,

extern void heap_inplace_update(Relation relation, HeapTuple tuple);
extern bool heap_prepare_freeze_tuple(HeapTupleHeader tuple,
TransactionId relfrozenxid, TransactionId relminmxid,
TransactionId cutoff_xid, TransactionId cutoff_multi,
const struct VacuumCutoffs *cutoffs,
HeapTupleFreeze *frz, bool *totally_frozen,
TransactionId *relfrozenxid_out,
MultiXactId *relminmxid_out);
Expand All @@ -188,9 +188,9 @@ extern void heap_freeze_execute_prepared(Relation rel, Buffer buffer,
HeapTupleFreeze *tuples, int ntuples);
extern bool heap_freeze_tuple(HeapTupleHeader tuple,
TransactionId relfrozenxid, TransactionId relminmxid,
TransactionId cutoff_xid, TransactionId cutoff_multi);
extern bool heap_tuple_would_freeze(HeapTupleHeader tuple, TransactionId cutoff_xid,
MultiXactId cutoff_multi,
TransactionId FreezeLimit, TransactionId MultiXactCutoff);
extern bool heap_tuple_would_freeze(HeapTupleHeader tuple,
const struct VacuumCutoffs *cutoffs,
TransactionId *relfrozenxid_out,
MultiXactId *relminmxid_out);
extern bool heap_tuple_needs_eventual_freeze(HeapTupleHeader tuple);
Expand Down
2 changes: 1 addition & 1 deletion src/include/access/tableam.h
Expand Up @@ -1634,7 +1634,7 @@ table_relation_copy_data(Relation rel, const RelFileLocator *newrlocator)
* in that index's order; if false and OldIndex is InvalidOid, no sorting is
* performed
* - OldIndex - see use_sort
* - OldestXmin - computed by vacuum_set_xid_limits(), even when
* - OldestXmin - computed by vacuum_get_cutoffs(), even when
* not needed for the relation's AM
* - *xid_cutoff - ditto
* - *multi_cutoff - ditto
Expand Down
49 changes: 42 additions & 7 deletions src/include/commands/vacuum.h
Expand Up @@ -235,6 +235,45 @@ typedef struct VacuumParams
int nworkers;
} VacuumParams;

/*
* VacuumCutoffs is immutable state that describes the cutoffs used by VACUUM.
* Established at the beginning of each VACUUM operation.
*/
struct VacuumCutoffs
{
/*
* Existing pg_class fields at start of VACUUM
*/
TransactionId relfrozenxid;
MultiXactId relminmxid;

/*
* OldestXmin is the Xid below which tuples deleted by any xact (that
* committed) should be considered DEAD, not just RECENTLY_DEAD.
*
* OldestMxact is the Mxid below which MultiXacts are definitely not seen
* as visible by any running transaction.
*
* OldestXmin and OldestMxact are also the most recent values that can
* ever be passed to vac_update_relstats() as frozenxid and minmulti
* arguments at the end of VACUUM. These same values should be passed
* when it turns out that VACUUM will leave no unfrozen XIDs/MXIDs behind
* in the table.
*/
TransactionId OldestXmin;
MultiXactId OldestMxact;

/*
* FreezeLimit is the Xid below which all Xids are definitely frozen or
* removed in pages VACUUM scans and cleanup locks.
*
* MultiXactCutoff is the value below which all MultiXactIds are
* definitely removed from Xmax in pages VACUUM scans and cleanup locks.
*/
TransactionId FreezeLimit;
MultiXactId MultiXactCutoff;
};

/*
* VacDeadItems stores TIDs whose index tuples are deleted by index vacuuming.
*/
Expand Down Expand Up @@ -286,13 +325,9 @@ extern void vac_update_relstats(Relation relation,
bool *frozenxid_updated,
bool *minmulti_updated,
bool in_outer_xact);
extern bool vacuum_set_xid_limits(Relation rel, const VacuumParams *params,
TransactionId *OldestXmin,
MultiXactId *OldestMxact,
TransactionId *FreezeLimit,
MultiXactId *MultiXactCutoff);
extern bool vacuum_xid_failsafe_check(TransactionId relfrozenxid,
MultiXactId relminmxid);
extern bool vacuum_get_cutoffs(Relation rel, const VacuumParams *params,
struct VacuumCutoffs *cutoffs);
extern bool vacuum_xid_failsafe_check(const struct VacuumCutoffs *cutoffs);
extern void vac_update_datfrozenxid(void);
extern void vacuum_delay_point(void);
extern bool vacuum_is_permitted_for_relation(Oid relid, Form_pg_class reltuple,
Expand Down

0 comments on commit 4ce3afb

Please sign in to comment.