Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/hash-and-trigram-name-indexes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"ensindexer": minor
"@ensnode/ensdb-sdk": minor
---

Re-enable `subgraph_domain.name` indexes (originally disabled in #1819) by pairing a hash index for exact-match lookups with a GIN trigram index (`gin_trgm_ops`) for partial-match filters (`_contains`, `_starts_with`, `_ends_with`). The hash index avoids the btree 8191-byte row size limit triggered by spam names. The trigram index requires the `pg_trgm` Postgres extension, which ENSIndexer now installs automatically via a Drizzle migration (`0001_enable_ext_pg_trgm.sql`) that runs before Ponder starts.
Comment thread
shrugs marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This guide covers running ENSNode locally for development and contributions.
### Prerequisites

- [Git](https://git-scm.com/)
- [Postgres 17](https://www.postgresql.org/)
- [Postgres 17](https://www.postgresql.org/) with the [`pg_trgm`](https://www.postgresql.org/docs/current/pgtrgm.html) extension available for installation (ships with stock Postgres contrib; ENSIndexer runs `CREATE EXTENSION IF NOT EXISTS pg_trgm` at startup to back partial-name search indexes)
- [Node.js](https://nodejs.org/)
- It's recommended you install Node.js through [nvm](https://github.com/nvm-sh/nvm) or [asdf](https://asdf-vm.com/).
- see `.nvmrc` and `.tool-versions` for the specific version of Node.js
Expand Down
4 changes: 4 additions & 0 deletions docs/ensnode.io/src/content/docs/docs/deploying/docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ import dockercompose from '@workspace/docker-compose.yml?raw';

The Docker images are the easiest way to run or deploy the ENSNode suite of services, both locally and in the cloud.

:::note[Postgres Requirement]
ENSIndexer runs `CREATE EXTENSION IF NOT EXISTS pg_trgm` at startup to back partial-name search indexes. If you're swapping out the bundled `postgres:17` image for a managed or custom Postgres, make sure the [`pg_trgm`](https://www.postgresql.org/docs/current/pgtrgm.html) extension is available for installation (it ships with stock Postgres contrib and is available for installation on most managed providers).
:::

<LinkCard
title="Configure Environment Variables"
href="/ensindexer/usage/configuration"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ These Terraform scripts are currently specific to ENSNode instances hosted by Na
- AWS account (for DNS management)
- AWS S3 bucket defined inside AWS account - `ensnode-terraform` (for Terraform state)

:::note[Postgres Requirement]
ENSIndexer runs `CREATE EXTENSION IF NOT EXISTS pg_trgm` at startup to back partial-name search indexes. Render's managed Postgres has [`pg_trgm`](https://www.postgresql.org/docs/current/pgtrgm.html) available by default; if you adapt this configuration to a different provider, confirm the extension is available for installation on your Postgres plan.
:::

## Configuration

Copy `.env.sample` to `.env.local` and fill in your configuration values:
Expand Down
4 changes: 4 additions & 0 deletions docs/ensnode.io/src/content/docs/docs/running/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ sidebar:

import { LinkCard } from '@astrojs/starlight/components';

:::note[Postgres Requirement]
ENSIndexer runs `CREATE EXTENSION IF NOT EXISTS pg_trgm` at startup to back partial-name search indexes. Any Postgres instance used as ENSDb must have the [`pg_trgm`](https://www.postgresql.org/docs/current/pgtrgm.html) extension available for installation (it ships with stock Postgres contrib and is available for installation on most managed providers).
:::

### Running in ens-test-env

`ens-test-env` provides a local Anvil chain with ENSNode, suitable for testing & developing ENS apps.
Expand Down
3 changes: 3 additions & 0 deletions packages/ensdb-sdk/migrations/0001_enable_ext_pg_trgm.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
-- This migration enables the pg_trgm extension, which is used for trigram-based indexing and
-- searching in PostgreSQL.
CREATE EXTENSION IF NOT EXISTS pg_trgm;
Comment thread
shrugs marked this conversation as resolved.
55 changes: 55 additions & 0 deletions packages/ensdb-sdk/migrations/meta/0001_snapshot.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"id": "7a9206c1-c22e-460d-8314-efa9b4cd64bc",
"prevId": "d661dcae-f64d-4ecd-a4da-3d5783e17e2c",
"version": "7",
"dialect": "postgresql",
"tables": {
"ensnode.metadata": {
"name": "metadata",
"schema": "ensnode",
"columns": {
"ens_indexer_schema_name": {
"name": "ens_indexer_schema_name",
"type": "text",
"primaryKey": false,
"notNull": true
},
"key": {
"name": "key",
"type": "text",
"primaryKey": false,
"notNull": true
},
"value": {
"name": "value",
"type": "jsonb",
"primaryKey": false,
"notNull": true
}
},
"indexes": {},
"foreignKeys": {},
"compositePrimaryKeys": {
"metadata_pkey": {
"name": "metadata_pkey",
"columns": ["ens_indexer_schema_name", "key"]
}
},
"uniqueConstraints": {},
"policies": {},
"checkConstraints": {},
"isRLSEnabled": false
}
},
"enums": {},
"schemas": {},
"views": {},
"sequences": {},
"roles": {},
"policies": {},
"_meta": {
"columns": {},
"schemas": {},
"tables": {}
}
}
7 changes: 7 additions & 0 deletions packages/ensdb-sdk/migrations/meta/_journal.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@
"when": 1773743108514,
"tag": "0000_cultured_captain_cross",
"breakpoints": true
},
{
"idx": 1,
"version": "7",
"when": 1776456213289,
"tag": "0001_enable_ext_pg_trgm",
"breakpoints": true
}
]
}
12 changes: 8 additions & 4 deletions packages/ensdb-sdk/src/ensindexer-abstract/subgraph.schema.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { Address } from "enssdk";
import { index, onchainTable, relations } from "ponder";
import { index, onchainTable, relations, sql } from "ponder";

import { monkeypatchCollate } from "../lib/collate";

Expand Down Expand Up @@ -93,9 +93,13 @@ export const subgraph_domain = onchainTable(
expiryDate: t.bigint(),
}),
(t) => ({
// Temporarily disable the `byName` index to avoid index creation issues.
// For more details, see: https://github.com/namehash/ensnode/issues/1819
// byName: index().on(t.name),
// uses a hash index because some name values exceed the btree max row size (8191 bytes)
Comment thread
shrugs marked this conversation as resolved.
Comment thread
shrugs marked this conversation as resolved.
byExactName: index().using("hash", t.name),
Comment thread
shrugs marked this conversation as resolved.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised no other code is updated in relation to us making the schema changes here? Ex: code in API handlers?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

none! postgres planner will use the indexes without specification

// GIN trigram index for partial-match filters (_contains, _starts_with, _ends_with).
// (inline `gin_trgm_ops` via `sql` because passing it through `.op()` gets dropped by Ponder,
// producing `USING gin (name)` with no opclass)
byFuzzyName: index().using("gin", sql`${t.name} gin_trgm_ops`),

byLabelhash: index().on(t.labelhash),
byParentId: index().on(t.parentId),
byOwnerId: index().on(t.ownerId),
Expand Down
Loading