Add additional database indexes for performance#1309
Add additional database indexes for performance#1309shangyian merged 9 commits intoDataJunction:mainfrom
Conversation
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
| __table_args__ = (UniqueConstraint("attribute_type_id", "column_id"),) | ||
| __table_args__ = ( | ||
| UniqueConstraint("attribute_type_id", "column_id"), | ||
| Index("idx_columnattribute_column_id", "column_id"), |
There was a problem hiding this comment.
I think that if the earlier UniqueConstraint had the column order switched then we wouldn't need this new one at all.
If it's easier to add then to change the other one then go for it.
There was a problem hiding this comment.
I ended up removing this because I'm not sure it makes much of a difference
| """ | ||
|
|
||
| __tablename__ = "nodeavailabilitystate" | ||
| __table_args__ = (Index("idx_nodeavailabilitystate_node_id", "node_id"),) |
| columns: Mapped[List["Column"]] = relationship( | ||
| back_populates="measure", | ||
| lazy="joined", | ||
| order_by="Column.name", |
There was a problem hiding this comment.
We don't, removed!
| """ | ||
|
|
||
| __tablename__ = "noderelationship" | ||
| __table_args__ = ( |
| foreign_keys="History.entity_name", | ||
| ) | ||
|
|
||
| __table_args__ = ( |
There was a problem hiding this comment.
I removed these changes
|
|
||
| __tablename__ = "partition" | ||
| __table_args__ = ( | ||
| Index("idx_partition_column_id", "column_id", postgresql_using="btree"), |
There was a problem hiding this comment.
I think btree is the default, so probably no need to specify it everywhere.
There was a problem hiding this comment.
Agreed, also removed these indexes because I don't think they make a difference for the specific slow endpoints I'm looking at
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
|
@agorajek I updated this with a smaller subset of indexes, which do show a stat sig increase in endpoint speed (see the updated PR description for details). |
2 participants
Summary
This migration adds missing indexes across several tables to improve query performance, particularly for recursive CTE queries and other common join patterns in the metadata database.
Specifically, indexes were added on foreign keys and frequently filtered columns across the following tables:
cube,dimensionlink,nodecolumns, andnoderelationship.Benchmark Results (100 runs each) on
/sql/measuresBenchmark Results (100 runs each) on
/metrics/common/dimensionsA two-sample t-test confirms the difference is statistically significant for both endpoints.
Test Plan
make checkpassesmake testshows 100% unit test coverageDeployment Plan