Skip to content

Commit

Permalink
Selectively invalidate RegClass and RegType
Browse files Browse the repository at this point in the history
RegClass is an easy choice, because those invalidations are also
the invalidations of TupleDescriptors, and because it has a nice
API; we are passed the oid of the relation to invalidate, so we
acquire the target in O(1).

(Note in passing: AttributeImpl is built on SwitchPointCache in
the pattern that's emerged for CatalogObjects in general, and an
AttributeImpl.Cataloged uses the SwitchPoint of the RegClass, so
it's clear that all the attributes of the associated tuple
descriptor will do the right thing upon invalidation. In contrast,
TupleDescriptorImpl itself isn't quite built that way, and the
question of just how a TupleDescriptor itself should act after
invalidation hasn't been fully nailed down yet.)

RegType is probably also worth invalidating selectively, as is
probably RegProcedure (procedures are mainly what we're about
in PL/Java. right?), though only RegType is done here.

That API is less convenient; we are passed not the oid but a hash
of the oid, and not the hash that Java uses. The solution here is
brute force, to get an initial working implementation. There are
plenty of opportunities for optimization.

One idea would be to use a subclass of SwitchPoint that would set
a flag, or invoke a Runnable, the first time its guardWithTest
method is called. If that hasn't happened, there is nothing to
invalidate. The Runnable could add the containing object into some
data structure more easily searched by the supplied hash. Transitions
of the data structure between empty and not-empty could be propagated
to a boolean in native memory, where the C callback code could avoid
the Java upcall entirely if there is nothing to do. This commit
contains none of those optimizations.

Factory.invalidateType might be misnamed; it could be syscacheInvalidate
and take the syscache id as another parameter, and then dispatch to
invalidating a RegType or RegProcedure or what have you, as the case
may be.

At least, that would be a more concise implementation than providing
separate Java methods and having the C callback decide which to call.
But if some later optimization is tracking anything-to-invalidate?
separately for them, then the C code might be the efficient place
for the check to be done.

PostgreSQL has a limited number of slots for invalidation callbacks,
and requires a separate registration (using another slot) for each
syscache id for which callbacks are wanted (even though you get
the affected syscache id in the callback?!). It would be antisocial
to grab one for every sort of CatalogObject supported here, so we
will have many relying on CatalogObject.Addressed.s_globalPoint
and some strategy for zapping that every so often. That is not
included in this commit. (The globalPoint exists, but there is
not yet anything that ever zaps it.)

Some imperfect strategy that isn't guaranteed conservative might
be necessary, and might be tolerable (PL/Java has existed for years
with less attention to invalidation). An early idea was to zap the
globalPoint on every transaction or subtransaction boundary, or when
the command counter has been incremented; those are times when
PostgreSQL processes invalidations. However, invalidations are also
processed any time locks are acquired, and that doesn't sound as if
it would be practical to intercept (or as if the resulting behavior
would be practical, even if it could be done).

Another solution approach would just be to expose a zapGlobalPoint
knob as API; if some code wants to be sure it is not seeing something
stale (in any CatalogObject we aren't doing selective invalidation for),
it can just say so before fetching it.
  • Loading branch information
jcflack committed Jan 23, 2022
1 parent e742fae commit 5adf2c8
Show file tree
Hide file tree
Showing 4 changed files with 205 additions and 5 deletions.
40 changes: 40 additions & 0 deletions pljava-so/src/main/c/ModelUtils.c
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@
* by these methods won't be shifting underneath them.
*/

static jclass s_CatalogObjectImpl_Factory_class;
static jmethodID s_CatalogObjectImpl_Factory_invalidateRelation;
static jmethodID s_CatalogObjectImpl_Factory_invalidateType;

static jclass s_MemoryContextImpl_class;
static jmethodID s_MemoryContextImpl_callback;
static void memoryContextCallback(void *arg);
Expand All @@ -71,6 +75,9 @@ static jclass s_TupleTableSlotImpl_class;
static jmethodID s_TupleTableSlotImpl_newDeformed;
static jmethodID s_TupleTableSlotImpl_supplyHeapTuples;

static void relCacheCB(Datum arg, Oid relid);
static void sysCacheCB(Datum arg, int cacheid, uint32 hash);

jobject pljava_TupleDescriptor_create(TupleDesc tupdesc, Oid reloid)
{
jlong tupdesc_size = (jlong)TupleDescSize(tupdesc);
Expand Down Expand Up @@ -139,6 +146,12 @@ static void memoryContextCallback(void *arg)
p2l.longVal);
}

static void relCacheCB(Datum arg, Oid relid)
{
JNI_callStaticObjectMethodLocked(s_CatalogObjectImpl_Factory_class,
s_CatalogObjectImpl_Factory_invalidateRelation, (jint)relid);
}

static void resourceReleaseCB(ResourceReleasePhase phase,
bool isCommit, bool isTopLevel, void *arg)
{
Expand Down Expand Up @@ -178,6 +191,19 @@ static void resourceReleaseCB(ResourceReleasePhase phase,
Backend_warnJEP411(isCommit);
}

static void sysCacheCB(Datum arg, int cacheid, uint32 hash)
{
switch ( cacheid )
{
case TYPEOID:
JNI_callStaticObjectMethodLocked(s_CatalogObjectImpl_Factory_class,
s_CatalogObjectImpl_Factory_invalidateType, (jint)hash);
break;
default:
break;
}
}

void pljava_ResourceOwner_unregister(void)
{
UnregisterResourceReleaseCallback(resourceReleaseCB, NULL);
Expand Down Expand Up @@ -331,6 +357,16 @@ void pljava_ModelUtils_initialize(void)
PgObject_registerNatives2(cls, catalogObjectAddressedMethods);
JNI_deleteLocalRef(cls);

cls = PgObject_getJavaClass("org/postgresql/pljava/pg/CatalogObjectImpl$Factory");
s_CatalogObjectImpl_Factory_class = JNI_newGlobalRef(cls);
JNI_deleteLocalRef(cls);
s_CatalogObjectImpl_Factory_invalidateRelation =
PgObject_getStaticJavaMethod(
s_CatalogObjectImpl_Factory_class, "invalidateRelation", "(I)V");
s_CatalogObjectImpl_Factory_invalidateType =
PgObject_getStaticJavaMethod(
s_CatalogObjectImpl_Factory_class, "invalidateType", "(I)V");

cls = PgObject_getJavaClass("org/postgresql/pljava/pg/CharsetEncodingImpl$EarlyNatives");
PgObject_registerNatives2(cls, charsetMethods);
JNI_deleteLocalRef(cls);
Expand Down Expand Up @@ -388,6 +424,10 @@ void pljava_ModelUtils_initialize(void)
"(Ljava/nio/ByteBuffer;)Ljava/util/List;");

RegisterResourceReleaseCallback(resourceReleaseCB, NULL);

CacheRegisterRelcacheCallback(relCacheCB, 0);

CacheRegisterSyscacheCallback(TYPEOID, sysCacheCB, 0);
}

/*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,70 @@ static Supplier<CatalogObjectImpl> ctorIfKnown(
reachabilityFence(fieldRead); // insist the read really happens
}
}

/**
* Called from native code with a relation oid when one relation's
* metadata has been invalidated, or with {@code InvalidOid} to flush
* all relation metadata.
*/
private static void invalidateRelation(int relOid)
{
assert threadMayEnterPG() : "RegClass invalidate thread";

List<SwitchPoint> sps = new ArrayList<>();
List<Runnable> postOps = new ArrayList<>();

if ( InvalidOid != relOid )
{
RegClassImpl c = (RegClassImpl)
findObjectId(RegClass.CLASSID, relOid);
if ( null != c )
c.invalidate(sps, postOps);
}
else // invalidate all RegClass instances
{
forEachValue(o ->
{
if ( o instanceof RegClassImpl )
((RegClassImpl)o).invalidate(sps, postOps);
});
}

if ( sps.isEmpty() )
return;

SwitchPoint.invalidateAll(sps.stream().toArray(SwitchPoint[]::new));

postOps.forEach(Runnable::run);
}

/**
* Called from native code with the {@code catcache} hash of the type
* Oid (inconvenient, as that is likely different from the hash Java
* uses), or zero to flush metadata for all cached types.
*/
private static void invalidateType(int oidHash)
{
assert threadMayEnterPG() : "RegType invalidate thread";

List<SwitchPoint> sps = new ArrayList<>();
List<Runnable> postOps = new ArrayList<>();

forEachValue(o ->
{
if ( ! ( o instanceof RegTypeImpl ) )
return;
if ( 0 == oidHash || oidHash == murmurhash32(o.oid()) )
((RegTypeImpl)o).invalidate(sps, postOps);
});

if ( sps.isEmpty() )
return;

SwitchPoint.invalidateAll(sps.stream().toArray(SwitchPoint[]::new));

postOps.forEach(Runnable::run);
}
}

/*
Expand Down Expand Up @@ -952,4 +1016,20 @@ static UnsupportedOperationException notyet(String what)
return new UnsupportedOperationException(
"CatalogObject API " + what);
}

/**
* The Oid hash function used by the backend's Oid-based catalog caches
* to identify the entries affected by invalidation events.
*<p>
* From hashutils.h.
*/
static int murmurhash32(int h)
{
h ^= h >>> 16;
h *= 0x85ebca6b;
h ^= h >>> 13;
h *= 0xc2b2ae35;
h ^= h >>> 16;
return h;
}
}
32 changes: 32 additions & 0 deletions pljava/src/main/java/org/postgresql/pljava/pg/RegClassImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,38 @@ int cacheId()
m_cacheSwitchPoint = new SwitchPoint();
}

/**
* Called from {@code Factory}'s {@code invalidateRelation} to set up
* the invalidation of this relation's metadata.
*<p>
* Adds this relation's {@code SwitchPoint} to the caller's list so that,
* if more than one is to be invalidated, that can be done in bulk. Adds to
* <var>postOps</var> any operations the caller should conclude with
* after invalidating the {@code SwitchPoint}.
*/
void invalidate(List<SwitchPoint> sps, List<Runnable> postOps)
{
TupleDescriptor.Interned[] oldTDH = m_tupDescHolder;
sps.add(m_cacheSwitchPoint);

/*
* Before invalidating the SwitchPoint, line up a new one (and a newly
* nulled tupDescHolder) for value-computing methods to find once the
* old SwitchPoint is invalidated.
*/
m_cacheSwitchPoint = new SwitchPoint();
m_tupDescHolder = null;

/*
* After the old SwitchPoint gets invalidated, the old tupDescHolder,
* if any, can have its element nulled so the old TupleDescriptor can
* be collected without having to wait for the 'guardWithTest's it is
* bound into to be recomputed.
*/
if ( null != oldTDH )
postOps.add(() -> oldTDH[0] = null);
}

/**
* Associated tuple descriptor, redundantly kept accessible here as well as
* opaquely bound into a {@code SwitchPointCache} method handle.
Expand Down
58 changes: 53 additions & 5 deletions pljava/src/main/java/org/postgresql/pljava/pg/RegTypeImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
import java.sql.SQLType;
import java.sql.SQLException;

import java.util.List;

import java.util.function.UnaryOperator;

import static org.postgresql.pljava.internal.SwitchPointCache.doNotCache;
Expand Down Expand Up @@ -61,12 +63,13 @@ abstract class RegTypeImpl extends Addressed<RegType>
AccessControlled<CatalogObject.USAGE>, RegType
{
/**
* For the time being, punt and return the global switch point.
* Per-instance switch point, to be invalidated selectively
* by a syscache callback.
*<p>
* Only {@link NoModifier NoModifier} carries one; derived instances of
* {@link Modified Modified} or {@link Blessed Blessed} return that one.
*/
SwitchPoint cacheSwitchPoint()
{
return s_globalPoint[0];
}
abstract SwitchPoint cacheSwitchPoint();

@Override
int cacheId()
Expand All @@ -83,6 +86,23 @@ int cacheId()
super(slots);
}

/**
* Called from {@code Factory}'s {@code invalidateType} to set up
* the invalidation of this type's metadata.
*<p>
* Adds this type's {@code SwitchPoint} to the caller's list so that,
* if more than one is to be invalidated, that can be done in bulk. Adds to
* <var>postOps</var> any operations the caller should conclude with
* after invalidating the {@code SwitchPoint}.
*/
void invalidate(List<SwitchPoint> sps, List<Runnable> postOps)
{
/*
* We don't expect invalidations for any flavor except NoModifier, so
* this no-op version will be overridden there only.
*/
}

/**
* Holder for the {@code RegClass} corresponding to {@code relation()},
* only non-null during a call of {@code dualHandshake}.
Expand Down Expand Up @@ -363,9 +383,25 @@ public int modifier()
*/
static class NoModifier extends RegTypeImpl
{
private SwitchPoint m_sp;

@Override
SwitchPoint cacheSwitchPoint()
{
return m_sp;
}

NoModifier()
{
super(s_initializer.apply(new MethodHandle[NSLOTS]));
m_sp = new SwitchPoint();
}

@Override
void invalidate(List<SwitchPoint> sps, List<Runnable> postOps)
{
sps.add(m_sp);
m_sp = new SwitchPoint();
}

@Override
Expand Down Expand Up @@ -400,6 +436,12 @@ static class Modified extends RegTypeImpl
{
private final NoModifier m_base;

@Override
SwitchPoint cacheSwitchPoint()
{
return m_base.m_sp;
}

Modified(NoModifier base)
{
super(base.m_slots);
Expand Down Expand Up @@ -467,6 +509,12 @@ static class Blessed extends RegTypeImpl
NSLOTS = i;
}

@Override
SwitchPoint cacheSwitchPoint()
{
return ((NoModifier)RECORD).m_sp;
}

Blessed()
{
super(((RegTypeImpl)RECORD).m_slots);
Expand Down

0 comments on commit 5adf2c8

Please sign in to comment.