Skip to content

Add Arrow Hash Index docs#1324

Merged
lukekim merged 3 commits into
release/1.11from
lukim/hash-index
Jan 19, 2026
Merged

Add Arrow Hash Index docs#1324
lukekim merged 3 commits into
release/1.11from
lukim/hash-index

Conversation

@lukekim
Copy link
Copy Markdown
Contributor

@lukekim lukekim commented Jan 16, 2026

This pull request introduces documentation for a new experimental "hash index" feature for Arrow-accelerated datasets, available in Spice v1.11.0-rc.2 and later. The hash index enables O(1) point lookups on primary key columns, improving query performance for equality predicates. The documentation covers configuration, supported data types, performance characteristics, memory usage, and limitations. References to this new feature are added throughout relevant docs, and guidance is provided on when and how to use it.

Hash Index Feature Documentation:

  • Added a comprehensive new page, hash-index.md, describing the experimental hash index for Arrow acceleration, including configuration, supported data types, performance, memory usage, architecture, and troubleshooting.
  • Updated arrow.md to introduce the hash index, provide configuration examples, and clarify that while traditional indexes are not supported, hash indexes are now available for point lookups.

Cross-References and Guidance:

  • Updated the general indexes documentation to clarify that traditional indexes are not supported for Arrow acceleration, but hash indexes are available as an experimental feature. Added cross-reference to the new documentation.
  • Added a row to the performance tuning guide, recommending Arrow with hash index for point lookups on small to medium datasets, and provided configuration details.

Memory Usage Documentation:

  • Updated the memory reference documentation to include details and a table on the additional memory overhead incurred by enabling the hash index, with concrete examples.

@lukekim lukekim added this to the v1.11 milestone Jan 16, 2026
@lukekim lukekim self-assigned this Jan 16, 2026
@lukekim lukekim added enhancement New feature or request area/docs labels Jan 16, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 16, 2026 •
edited
Loading

✅ Pull with Spice Passed

Passing checks:

  • ✅ Title meets minimum length requirement (10 characters)
  • ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
  • ✅ No banned labels detected
  • ✅ Has at least one assignee: lukekim

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 16, 2026

🚀 deployed to https://1dd59dbd.spiceai-org-website.pages.dev

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 16, 2026

🚀 deployed to https://d8f3cb48.spiceai-org-website.pages.dev

@lukekim lukekim merged commit cbfb678 into release/1.11 Jan 19, 2026
3 checks passed
@lukekim lukekim deleted the lukim/hash-index branch January 19, 2026 23:29
Jeadie added a commit that referenced this pull request Jan 20, 2026
@lukekim @phillipleblanc
@Jeadie @ewgenius @krinart @sgrebnov
v1.11 Documentation (#1296)
5af4834 
* Remove OTel ports from docs (#1270)

* Docs for Google (#1286)

Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>

* Parameterized Queries docs (#1298)

* Paramterized Queries docs

* Formatting

* Add SMB and NFS Data Connector Docs (#1295)

* Add SMB & NFS Data Connector docs

* Fixes

* formatting

* Rename "params" key (#1300)

* Fix params (#1301)

* Update Dynamodb Authentication (#1304)

* Update Dynamodb Auth

* Update

* Cayenne: document cayenne_file_path and cayenne_metadata_dir (#1307)

* Update snapshots documentation (#1318)

* Update snapshots documentation

* Fix

* Docs for snapshots_reset_expiry_on_load (#1322)

* Minor fixes for DynamoDB (#1323)

* Minor fixes for DynamoDB

* Minor fix

* Update Distributed Query docs for v1.11 changes (#1326)

* Update Distributed Query docs for v1.11 changes

* Update website/docs/features/distributed-query/index.md

Co-authored-by: Jack Eadie <jack@spice.ai>

---------

Co-authored-by: Jack Eadie <jack@spice.ai>

* ScyllaDB Data Connector docs (#1325)

* Add Arrow Hash Index docs (#1324)

* Add Arrow Hash Index docs

* Formatting

* Add versioning support (#1308)

* Add versioning support

* Fix: empty versions array until release branches exist

* Enable versioned docs for release/1.11 branch

* Fix: resolve git refs with origin/ prefix for CI

* Fix: run git archive from repo root

* Fix: serve current docs at /docs, versioned docs at /docs/v1.11

* fix(versioning): only show unmaintained banner for versions outside maintenance window

* Update to 1

* feat(versioning): add enterprise support note on unmaintained version banner

* feat(versioning): add support for unreleased docs from trunk

- Current docs (trunk) now served at /docs/next with 'unreleased' banner
- Latest release branch served at /docs as the default
- Previous versions continue at /docs/v1.11, etc.

* Move to right

* feat(versioning): auto-detect release branches from git

- Script now auto-detects release/<major>.<minor> branches
- No manual VERSIONS array maintenance required
- Creating a new release branch automatically adds version to docs

* Fix

* fix(versioning): highest version is 'next', second highest is 'latest'

- v1.11.x (highest) → Next (unreleased) at /docs/next
- v1.10.x (second) → Latest (stable) at /docs
- Trunk docs available at /docs/trunk
- Previous versions at /docs/v1.9, etc.

* fix(versioning): warn on broken links for cross-version compatibility

Older release branches may contain absolute links to docs pages that
don't exist in all versions. Setting onBrokenLinks to 'warn' allows
the build to succeed while still reporting these issues.

* Fixes

* Fixes

* Update menu

* Improve SMB docs (#1328)

* Improve copilot instructions

* Improve SMB docs

---------

Co-authored-by: Phillip LeBlanc <phillip@spice.ai>
Co-authored-by: Jack Eadie <jack@spice.ai>
Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Viktor Yershov <viktor@spice.ai>
Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com>
Jeadie added a commit that referenced this pull request Jan 26, 2026
@Jeadie @lukekim
@phillipleblanc @ewgenius @krinart @sgrebnov
Update snapshot storage configuration description (#1331)
8db3b0e 
* v1.11 Documentation (#1296)

* Remove OTel ports from docs (#1270)

* Docs for Google (#1286)

Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>

* Parameterized Queries docs (#1298)

* Paramterized Queries docs

* Formatting

* Add SMB and NFS Data Connector Docs (#1295)

* Add SMB & NFS Data Connector docs

* Fixes

* formatting

* Rename "params" key (#1300)

* Fix params (#1301)

* Update Dynamodb Authentication (#1304)

* Update Dynamodb Auth

* Update

* Cayenne: document cayenne_file_path and cayenne_metadata_dir (#1307)

* Update snapshots documentation (#1318)

* Update snapshots documentation

* Fix

* Docs for snapshots_reset_expiry_on_load (#1322)

* Minor fixes for DynamoDB (#1323)

* Minor fixes for DynamoDB

* Minor fix

* Update Distributed Query docs for v1.11 changes (#1326)

* Update Distributed Query docs for v1.11 changes

* Update website/docs/features/distributed-query/index.md

Co-authored-by: Jack Eadie <jack@spice.ai>

---------

Co-authored-by: Jack Eadie <jack@spice.ai>

* ScyllaDB Data Connector docs (#1325)

* Add Arrow Hash Index docs (#1324)

* Add Arrow Hash Index docs

* Formatting

* Add versioning support (#1308)

* Add versioning support

* Fix: empty versions array until release branches exist

* Enable versioned docs for release/1.11 branch

* Fix: resolve git refs with origin/ prefix for CI

* Fix: run git archive from repo root

* Fix: serve current docs at /docs, versioned docs at /docs/v1.11

* fix(versioning): only show unmaintained banner for versions outside maintenance window

* Update to 1

* feat(versioning): add enterprise support note on unmaintained version banner

* feat(versioning): add support for unreleased docs from trunk

- Current docs (trunk) now served at /docs/next with 'unreleased' banner
- Latest release branch served at /docs as the default
- Previous versions continue at /docs/v1.11, etc.

* Move to right

* feat(versioning): auto-detect release branches from git

- Script now auto-detects release/<major>.<minor> branches
- No manual VERSIONS array maintenance required
- Creating a new release branch automatically adds version to docs

* Fix

* fix(versioning): highest version is 'next', second highest is 'latest'

- v1.11.x (highest) → Next (unreleased) at /docs/next
- v1.10.x (second) → Latest (stable) at /docs
- Trunk docs available at /docs/trunk
- Previous versions at /docs/v1.9, etc.

* fix(versioning): warn on broken links for cross-version compatibility

Older release branches may contain absolute links to docs pages that
don't exist in all versions. Setting onBrokenLinks to 'warn' allows
the build to succeed while still reporting these issues.

* Fixes

* Fixes

* Update menu

* Improve SMB docs (#1328)

* Improve copilot instructions

* Improve SMB docs

---------

Co-authored-by: Phillip LeBlanc <phillip@spice.ai>
Co-authored-by: Jack Eadie <jack@spice.ai>
Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Viktor Yershov <viktor@spice.ai>
Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com>

* Update snapshot storage configuration description

Clarified that the location for snapshots must be an S3 directory instead of a folder.

---------

Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>
Co-authored-by: Phillip LeBlanc <phillip@spice.ai>
Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Viktor Yershov <viktor@spice.ai>
Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com>
lukekim added a commit that referenced this pull request Jan 28, 2026
@lukekim @phillipleblanc
@Jeadie @ewgenius @krinart @sgrebnov
Merge v1.11 docs to trunk and set v2.0 as next (#1345)
166dced 
* Remove OTel ports from docs (#1270)

* Docs for Google (#1286)

Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>

* Parameterized Queries docs (#1298)

* Paramterized Queries docs

* Formatting

* Add SMB and NFS Data Connector Docs (#1295)

* Add SMB & NFS Data Connector docs

* Fixes

* formatting

* Rename "params" key (#1300)

* Fix params (#1301)

* Update Dynamodb Authentication (#1304)

* Update Dynamodb Auth

* Update

* Cayenne: document cayenne_file_path and cayenne_metadata_dir (#1307)

* Update snapshots documentation (#1318)

* Update snapshots documentation

* Fix

* Docs for snapshots_reset_expiry_on_load (#1322)

* Minor fixes for DynamoDB (#1323)

* Minor fixes for DynamoDB

* Minor fix

* Update Distributed Query docs for v1.11 changes (#1326)

* Update Distributed Query docs for v1.11 changes

* Update website/docs/features/distributed-query/index.md

Co-authored-by: Jack Eadie <jack@spice.ai>

---------

Co-authored-by: Jack Eadie <jack@spice.ai>

* ScyllaDB Data Connector docs (#1325)

* Add Arrow Hash Index docs (#1324)

* Add Arrow Hash Index docs

* Formatting

* Add versioning support (#1308)

* Add versioning support

* Fix: empty versions array until release branches exist

* Enable versioned docs for release/1.11 branch

* Fix: resolve git refs with origin/ prefix for CI

* Fix: run git archive from repo root

* Fix: serve current docs at /docs, versioned docs at /docs/v1.11

* fix(versioning): only show unmaintained banner for versions outside maintenance window

* Update to 1

* feat(versioning): add enterprise support note on unmaintained version banner

* feat(versioning): add support for unreleased docs from trunk

- Current docs (trunk) now served at /docs/next with 'unreleased' banner
- Latest release branch served at /docs as the default
- Previous versions continue at /docs/v1.11, etc.

* Move to right

* feat(versioning): auto-detect release branches from git

- Script now auto-detects release/<major>.<minor> branches
- No manual VERSIONS array maintenance required
- Creating a new release branch automatically adds version to docs

* Fix

* fix(versioning): highest version is 'next', second highest is 'latest'

- v1.11.x (highest) → Next (unreleased) at /docs/next
- v1.10.x (second) → Latest (stable) at /docs
- Trunk docs available at /docs/trunk
- Previous versions at /docs/v1.9, etc.

* fix(versioning): warn on broken links for cross-version compatibility

Older release branches may contain absolute links to docs pages that
don't exist in all versions. Setting onBrokenLinks to 'warn' allows
the build to succeed while still reporting these issues.

* Fixes

* Fixes

* Update menu

* Improve SMB docs (#1328)

* Improve copilot instructions

* Improve SMB docs

* Docs for snapshots_creation_policy (#1330)

* Improvements (#1333)

* Add High Availability documentation for distributed query clusters (#1334)

* Update kafka/debezium docs

* Update snapshot storage configuration description (#1331)

* v1.11 Documentation (#1296)

* Remove OTel ports from docs (#1270)

* Docs for Google (#1286)

Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>

* Parameterized Queries docs (#1298)

* Paramterized Queries docs

* Formatting

* Add SMB and NFS Data Connector Docs (#1295)

* Add SMB & NFS Data Connector docs

* Fixes

* formatting

* Rename "params" key (#1300)

* Fix params (#1301)

* Update Dynamodb Authentication (#1304)

* Update Dynamodb Auth

* Update

* Cayenne: document cayenne_file_path and cayenne_metadata_dir (#1307)

* Update snapshots documentation (#1318)

* Update snapshots documentation

* Fix

* Docs for snapshots_reset_expiry_on_load (#1322)

* Minor fixes for DynamoDB (#1323)

* Minor fixes for DynamoDB

* Minor fix

* Update Distributed Query docs for v1.11 changes (#1326)

* Update Distributed Query docs for v1.11 changes

* Update website/docs/features/distributed-query/index.md

Co-authored-by: Jack Eadie <jack@spice.ai>

---------

Co-authored-by: Jack Eadie <jack@spice.ai>

* ScyllaDB Data Connector docs (#1325)

* Add Arrow Hash Index docs (#1324)

* Add Arrow Hash Index docs

* Formatting

* Add versioning support (#1308)

* Add versioning support

* Fix: empty versions array until release branches exist

* Enable versioned docs for release/1.11 branch

* Fix: resolve git refs with origin/ prefix for CI

* Fix: run git archive from repo root

* Fix: serve current docs at /docs, versioned docs at /docs/v1.11

* fix(versioning): only show unmaintained banner for versions outside maintenance window

* Update to 1

* feat(versioning): add enterprise support note on unmaintained version banner

* feat(versioning): add support for unreleased docs from trunk

- Current docs (trunk) now served at /docs/next with 'unreleased' banner
- Latest release branch served at /docs as the default
- Previous versions continue at /docs/v1.11, etc.

* Move to right

* feat(versioning): auto-detect release branches from git

- Script now auto-detects release/<major>.<minor> branches
- No manual VERSIONS array maintenance required
- Creating a new release branch automatically adds version to docs

* Fix

* fix(versioning): highest version is 'next', second highest is 'latest'

- v1.11.x (highest) → Next (unreleased) at /docs/next
- v1.10.x (second) → Latest (stable) at /docs
- Trunk docs available at /docs/trunk
- Previous versions at /docs/v1.9, etc.

* fix(versioning): warn on broken links for cross-version compatibility

Older release branches may contain absolute links to docs pages that
don't exist in all versions. Setting onBrokenLinks to 'warn' allows
the build to succeed while still reporting these issues.

* Fixes

* Fixes

* Update menu

* Improve SMB docs (#1328)

* Improve copilot instructions

* Improve SMB docs

---------

Co-authored-by: Phillip LeBlanc <phillip@spice.ai>
Co-authored-by: Jack Eadie <jack@spice.ai>
Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Viktor Yershov <viktor@spice.ai>
Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com>

* Update snapshot storage configuration description

Clarified that the location for snapshots must be an S3 directory instead of a folder.

---------

Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>
Co-authored-by: Phillip LeBlanc <phillip@spice.ai>
Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Viktor Yershov <viktor@spice.ai>
Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com>

* Update Snowflake data connector docs (#1340)

* fix: convert absolute /docs/ links to relative paths

* Update Cayenne accelerator status from Alpha to Beta (#1341)

* URL tables docs (#1343)

* URL tables docs

* Formatting

* Improve snapshots documentation + retention (#1342)

* Improve snapshots documnetation + retention

* Fix

---------

Co-authored-by: Jack Eadie <jack@spice.ai>

* Fix links

* Formatting

* Update version references in documentation and scripts for v2.1 release

* Handle major versions

* Fix merge conflicts

---------

Co-authored-by: Phillip LeBlanc <phillip@spice.ai>
Co-authored-by: Jack Eadie <jack@spice.ai>
Co-authored-by: Evgenii Khramkov <evgenii@spice.ai>
Co-authored-by: Viktor Yershov <viktor@spice.ai>
Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@phillipleblanc phillipleblanc phillipleblanc approved these changes

@sgrebnov sgrebnov sgrebnov approved these changes

Assignees

@lukekim lukekim

Labels

area/docs enhancement New feature or request

Projects

None yet

Milestone

v1.11

Development

Successfully merging this pull request may close these issues.

3 participants

@lukekim @phillipleblanc @sgrebnov