Skip to content

Update vector search index test fixtures based on cloud #5366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
janniklasrose merged 8 commits into from
Jun 2, 2026
+214 −125
Merged

Update vector search index test fixtures based on cloud #5366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f3da03f
Update test fixtures based on cloud
janniklasrose May 28, 2026
2a10ba5
Change recreate index_type test to embedding_dimension
janniklasrose May 31, 2026
fa71800
Use DIRECT_ACCESS instead of DELTA_SYNC
janniklasrose May 31, 2026
797aeaa
Validate index name in testserver
janniklasrose May 31, 2026
87ed778
Update unit test with correct index naming
janniklasrose Jun 1, 2026
f460274
Mark vector search index tests as CloudSlow
janniklasrose Jun 1, 2026
132adbe
Merge branch 'main' into janniklasrose/fix-vector-search-index-nightly
janniklasrose Jun 2, 2026
e9163fc
Update o= vs w= tests
janniklasrose Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion acceptance/bundle/deployment/bind/vector_search_index/databricks.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,7 @@ resources:
primary_key: id
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"columns":[{"name":"id","type":"integer"}]}'
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
1 change: 1 addition & 0 deletions acceptance/bundle/deployment/bind/vector_search_index/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion acceptance/bundle/deployment/bind/vector_search_index/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"endpoint_type": "STANDARD"
}

>>> [CLI] vector-search-indexes create-index --json {"name":"main.default.test_vs_index_[UNIQUE_NAME]","endpoint_name":"test-vs-endpoint-[UNIQUE_NAME]","primary_key":"id","index_type":"DIRECT_ACCESS","direct_access_index_spec":{"schema_json":"{\"columns\":[{\"name\":\"id\",\"type\":\"integer\"}]}"}}
>>> [CLI] vector-search-indexes create-index --json {"name":"main.default.test_vs_index_[UNIQUE_NAME]","endpoint_name":"test-vs-endpoint-[UNIQUE_NAME]","primary_key":"id","index_type":"DIRECT_ACCESS","direct_access_index_spec":{"schema_json":"{\"id\":\"integer\",\"vector\":\"array<float>\"}","embedding_vector_columns":[{"name":"vector","embedding_dimension":768}]}}
{
"name": "main.default.test_vs_index_[UNIQUE_NAME]",
"endpoint_name": "test-vs-endpoint-[UNIQUE_NAME]",
Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/deployment/bind/vector_search_index/script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ trap cleanup EXIT

trace $CLI vector-search-endpoints create-endpoint "${ENDPOINT_NAME}" STANDARD | jq '{name, endpoint_type}'

trace $CLI vector-search-indexes create-index --json "{\"name\":\"${INDEX_NAME}\",\"endpoint_name\":\"${ENDPOINT_NAME}\",\"primary_key\":\"id\",\"index_type\":\"DIRECT_ACCESS\",\"direct_access_index_spec\":{\"schema_json\":\"{\\\"columns\\\":[{\\\"name\\\":\\\"id\\\",\\\"type\\\":\\\"integer\\\"}]}\"}}" | jq '{name, endpoint_name, index_type, primary_key}'
trace $CLI vector-search-indexes create-index --json "{\"name\":\"${INDEX_NAME}\",\"endpoint_name\":\"${ENDPOINT_NAME}\",\"primary_key\":\"id\",\"index_type\":\"DIRECT_ACCESS\",\"direct_access_index_spec\":{\"schema_json\":\"{\\\"id\\\":\\\"integer\\\",\\\"vector\\\":\\\"array<float>\\\"}\",\"embedding_vector_columns\":[{\"name\":\"vector\",\"embedding_dimension\":768}]}}" | jq '{name, endpoint_name, index_type, primary_key}'

trace $CLI bundle deployment bind index1 "${INDEX_NAME}" --auto-approve

Expand Down
3 changes: 3 additions & 0 deletions acceptance/bundle/deployment/bind/vector_search_index/test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ Local = true
Cloud = true
RequiresUnityCatalog = true

# Vector Search indexes are slow to create, so only run them in the nightly (non-short) cloud job.
CloudSlow = true

Ignore = [
".databricks",
"databricks.yml",
Expand Down
12 changes: 7 additions & 5 deletions acceptance/bundle/invariant/configs/vector_search_index.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,12 @@ resources:
endpoint_type: STANDARD
vector_search_indexes:
foo:
name: test-index-$UNIQUE_NAME
name: main.default.test_index_$UNIQUE_NAME
endpoint_name: ${resources.vector_search_endpoints.bar.name}
primary_key: id
index_type: DELTA_SYNC
delta_sync_index_spec:
source_table: main.default.source_$UNIQUE_NAME
pipeline_type: TRIGGERED
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
12 changes: 7 additions & 5 deletions acceptance/bundle/resources/vector_search_indexes/basic/databricks.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ resources:
endpoint_type: STANDARD
vector_search_indexes:
my_index:
name: vs-index-$UNIQUE_NAME
name: main.default.vs_index_$UNIQUE_NAME
endpoint_name: ${resources.vector_search_endpoints.my_endpoint.name}
primary_key: id
index_type: DELTA_SYNC
delta_sync_index_spec:
source_table: main.default.source_$UNIQUE_NAME
pipeline_type: TRIGGERED
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
15 changes: 10 additions & 5 deletions acceptance/bundle/resources/vector_search_indexes/basic/out.requests.direct.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,18 @@
"method": "POST",
"path": "/api/2.0/vector-search/indexes",
"body": {
"delta_sync_index_spec": {
"pipeline_type": "TRIGGERED",
"source_table": "main.default.source_[UNIQUE_NAME]"
"direct_access_index_spec": {
"embedding_vector_columns": [
{
"embedding_dimension": 768,
"name": "vector"
}
],
"schema_json": "{\"id\":\"integer\",\"vector\":\"array<float>\"}"
},
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DELTA_SYNC",
"name": "vs-index-[UNIQUE_NAME]",
"index_type": "DIRECT_ACCESS",
"name": "main.default.vs_index_[UNIQUE_NAME]",
"primary_key": "id"
}
}
1 change: 1 addition & 0 deletions acceptance/bundle/resources/vector_search_indexes/basic/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 7 additions & 7 deletions acceptance/bundle/resources/vector_search_indexes/basic/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,20 @@ Resources:
URL: [DATABRICKS_URL]/compute/vector-search/vs-endpoint-[UNIQUE_NAME]?w=[NUMID]
Vector Search Indexes:
my_index:
Name: vs-index-[UNIQUE_NAME]
URL: (not deployed)
Name: main.default.vs_index_[UNIQUE_NAME]
URL: [DATABRICKS_URL]/explore/data/main/default/vs_index_[UNIQUE_NAME]?w=[NUMID]

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/deploy-vs-index-[UNIQUE_NAME]/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!

>>> [CLI] vector-search-indexes get-index vs-index-[UNIQUE_NAME]
>>> [CLI] vector-search-indexes get-index main.default.vs_index_[UNIQUE_NAME]
{
"name": "vs-index-[UNIQUE_NAME]",
"name": "main.default.vs_index_[UNIQUE_NAME]",
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DELTA_SYNC",
"index_type": "DIRECT_ACCESS",
"primary_key": "id"
}

Expand All @@ -51,8 +51,8 @@ Resources:
URL: [DATABRICKS_URL]/compute/vector-search/vs-endpoint-[UNIQUE_NAME]?w=[NUMID]
Vector Search Indexes:
my_index:
Name: vs-index-[UNIQUE_NAME]
URL: (not deployed)
Name: main.default.vs_index_[UNIQUE_NAME]
URL: [DATABRICKS_URL]/explore/data/main/default/vs_index_[UNIQUE_NAME]?w=[NUMID]

>>> print_requests.py //vector-search/indexes

Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/resources/vector_search_indexes/basic/script
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ rm -f out.requests.txt
trace $CLI bundle deploy

# Get index details
index_name="vs-index-${UNIQUE_NAME}"
index_name="main.default.vs_index_${UNIQUE_NAME}"
trace $CLI vector-search-indexes get-index "${index_name}" | jq '{name, endpoint_name, index_type, primary_key}'

trace $CLI bundle summary
Expand Down
12 changes: 7 additions & 5 deletions acceptance/bundle/resources/vector_search_indexes/drift/deleted_remotely/databricks.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ resources:
endpoint_type: STANDARD
vector_search_indexes:
my_index:
name: vs-index-$UNIQUE_NAME
name: main.default.vs_index_$UNIQUE_NAME
endpoint_name: ${resources.vector_search_endpoints.my_endpoint.name}
primary_key: id
index_type: DELTA_SYNC
delta_sync_index_spec:
source_table: main.default.source_$UNIQUE_NAME
pipeline_type: TRIGGERED
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
1 change: 1 addition & 0 deletions acceptance/bundle/resources/vector_search_indexes/drift/deleted_remotely/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions acceptance/bundle/resources/vector_search_indexes/drift/deleted_remotely/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Updating deployment state...
Deployment complete!

=== Simulate remote deletion
>>> [CLI] vector-search-indexes delete-index vs-index-[UNIQUE_NAME]
>>> [CLI] vector-search-indexes delete-index main.default.vs_index_[UNIQUE_NAME]

=== Plan detects missing resource and proposes creation
>>> [CLI] bundle plan
Expand All @@ -22,11 +22,11 @@ Deploying resources...
Updating deployment state...
Deployment complete!

>>> [CLI] vector-search-indexes get-index vs-index-[UNIQUE_NAME]
>>> [CLI] vector-search-indexes get-index main.default.vs_index_[UNIQUE_NAME]
{
"name": "vs-index-[UNIQUE_NAME]",
"name": "main.default.vs_index_[UNIQUE_NAME]",
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DELTA_SYNC",
"index_type": "DIRECT_ACCESS",
"primary_key": "id"
}

Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/resources/vector_search_indexes/drift/deleted_remotely/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
envsubst < databricks.yml.tmpl > databricks.yml
index_name="vs-index-${UNIQUE_NAME}"
index_name="main.default.vs_index_${UNIQUE_NAME}"

cleanup() {
trace $CLI bundle destroy --auto-approve
Expand Down
2 changes: 2 additions & 0 deletions acceptance/bundle/resources/vector_search_indexes/drift/deleted_remotely/test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
Cloud = false
# The parent sets CloudSlow=true; disable it here too, otherwise CloudSlow would imply Cloud=true.
CloudSlow = false
12 changes: 7 additions & 5 deletions ...ptance/bundle/resources/vector_search_indexes/drift/orphaned_endpoint/databricks.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ resources:
endpoint_type: STANDARD
vector_search_indexes:
my_index:
name: vs-index-$UNIQUE_NAME
name: main.default.vs_index_$UNIQUE_NAME
endpoint_name: ${resources.vector_search_endpoints.vs_endpoint.name}
primary_key: id
index_type: DELTA_SYNC
delta_sync_index_spec:
source_table: main.default.source_$UNIQUE_NAME
pipeline_type: TRIGGERED
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
1 change: 1 addition & 0 deletions acceptance/bundle/resources/vector_search_indexes/drift/orphaned_endpoint/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions acceptance/bundle/resources/vector_search_indexes/drift/orphaned_endpoint/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ Deploying resources...
Updating deployment state...
Deployment complete!

>>> [CLI] vector-search-indexes get-index vs-index-[UNIQUE_NAME]
>>> [CLI] vector-search-indexes get-index main.default.vs_index_[UNIQUE_NAME]
{
"name": "vs-index-[UNIQUE_NAME]",
"name": "main.default.vs_index_[UNIQUE_NAME]",
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DELTA_SYNC",
"index_type": "DIRECT_ACCESS",
"primary_key": "id"
}

Expand Down
2 changes: 1 addition & 1 deletion acceptance/bundle/resources/vector_search_indexes/drift/orphaned_endpoint/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
envsubst < databricks.yml.tmpl > databricks.yml
endpoint_name="vs-endpoint-${UNIQUE_NAME}"
index_name="vs-index-${UNIQUE_NAME}"
index_name="main.default.vs_index_${UNIQUE_NAME}"

cleanup() {
trace $CLI bundle destroy --auto-approve
Expand Down
2 changes: 2 additions & 0 deletions acceptance/bundle/resources/vector_search_indexes/drift/orphaned_endpoint/test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
Cloud = false
# The parent sets CloudSlow=true; disable it here too, otherwise CloudSlow would imply Cloud=true.
CloudSlow = false
5 changes: 4 additions & 1 deletion acceptance/bundle/resources/vector_search_indexes/grants/select/databricks.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@ resources:
primary_key: id
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"columns":[{"name":"id","type":"integer"}]}'
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
grants:
- principal: deco-test-user@databricks.com
privileges:
Expand Down
8 changes: 7 additions & 1 deletion acceptance/bundle/resources/vector_search_indexes/grants/select/out.requests.direct.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@
"path": "/api/2.0/vector-search/indexes",
"body": {
"direct_access_index_spec": {
"schema_json": "{\"columns\":[{\"name\":\"id\",\"type\":\"integer\"}]}"
"embedding_vector_columns": [
{
"embedding_dimension": 768,
"name": "vector"
}
],
"schema_json": "{\"id\":\"integer\",\"vector\":\"array<float>\"}"
},
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DIRECT_ACCESS",
Expand Down
1 change: 1 addition & 0 deletions acceptance/bundle/resources/vector_search_indexes/grants/select/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 7 additions & 5 deletions ...s/recreate/index_type/databricks.yml.tmpl → ...e/embedding_dimension/databricks.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ resources:
endpoint_type: STANDARD
vector_search_indexes:
my_index:
name: vs-index-$UNIQUE_NAME
name: main.default.vs_index_$UNIQUE_NAME
endpoint_name: ${resources.vector_search_endpoints.my_endpoint.name}
primary_key: id
index_type: DELTA_SYNC
delta_sync_index_spec:
source_table: main.default.source_$UNIQUE_NAME
pipeline_type: TRIGGERED
index_type: DIRECT_ACCESS
direct_access_index_spec:
schema_json: '{"id":"integer","vector":"array<float>"}'
embedding_vector_columns:
- name: vector
embedding_dimension: 768
23 changes: 23 additions & 0 deletions ...ources/vector_search_indexes/recreate/embedding_dimension/out.requests.create.direct.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"method": "POST",
"path": "/api/2.0/vector-search/indexes",
"body": {
"direct_access_index_spec": {
"embedding_vector_columns": [
{
"embedding_dimension": 768,
"name": "vector"
}
],
"schema_json": "{\"id\":\"integer\",\"vector\":\"array<float>\"}"
},
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DIRECT_ACCESS",
"name": "main.default.vs_index_[UNIQUE_NAME]",
"primary_key": "id"
}
}
{
"method": "GET",
"path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
}
35 changes: 35 additions & 0 deletions ...rces/vector_search_indexes/recreate/embedding_dimension/out.requests.recreate.direct.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"method": "GET",
"path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
}
{
"method": "DELETE",
"path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
}
{
"method": "GET",
"path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
}
{
"method": "POST",
"path": "/api/2.0/vector-search/indexes",
"body": {
"direct_access_index_spec": {
"embedding_vector_columns": [
{
"embedding_dimension": 384,
"name": "vector"
}
],
"schema_json": "{\"id\":\"integer\",\"vector\":\"array<float>\"}"
},
"endpoint_name": "vs-endpoint-[UNIQUE_NAME]",
"index_type": "DIRECT_ACCESS",
"name": "main.default.vs_index_[UNIQUE_NAME]",
"primary_key": "id"
}
}
{
"method": "GET",
"path": "/api/2.0/vector-search/indexes/main.default.vs_index_[UNIQUE_NAME]"
}
1 change: 1 addition & 0 deletions ...indexes/recreate/index_type/out.test.toml → ...ecreate/embedding_dimension/out.test.toml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading