Skip to content

Conversation

@rohanshah18
Copy link
Contributor

@rohanshah18 rohanshah18 commented Nov 12, 2025

Problem

Add fetch and update by metadata

Solution

This PR adds the fetchByMetadata and updateByMetadata methods for both Index and AsyncIndex clients, enabling fetching and updating vectors by metadata filters.

Fetch By Metadata
Basic Fetch: Fetch vectors matching a metadata filter with optional limit and pagination:

import io.pinecone.proto.FetchByMetadataResponse;
import com.google.protobuf.Struct;
import com.google.protobuf.Value;

// Create a metadata filter
Struct filter = Struct.newBuilder()
    .putFields("genre", Value.newBuilder()
        .setStructValue(Struct.newBuilder()
            .putFields("$eq", Value.newBuilder()
                .setStringValue("action")
                .build()))
        .build())
    .build();

// Fetch vectors by metadata with limit
FetchByMetadataResponse response = index.fetchByMetadata("example-namespace", filter, 10, null);
assertNotNull(response);
assertTrue(response.getVectorsCount() > 0);

// Access fetched vectors
response.getVectorsMap().forEach((id, vector) -> {
    assertNotNull(vector);
    assertTrue(vector.hasMetadata());
});

Pagination Support: Use pagination tokens to fetch additional pages of results:

// Fetch first page
FetchByMetadataResponse firstPage = index.fetchByMetadata("example-namespace", filter, 2, null);
assertNotNull(firstPage);

// Fetch next page using pagination token
if (firstPage.hasPagination() && 
    firstPage.getPagination().getNext() != null && 
    !firstPage.getPagination().getNext().isEmpty()) {
    
    FetchByMetadataResponse secondPage = index.fetchByMetadata(
        "example-namespace", filter, 2, firstPage.getPagination().getNext());
    assertNotNull(secondPage);
}

Update By Metadata
Basic Update: Update vectors matching a metadata filter with new metadata:

import io.pinecone.proto.UpdateResponse;
import com.google.protobuf.Struct;

// Create a filter to match vectors
Struct filter = Struct.newBuilder()
    .putFields("genre", Value.newBuilder()
        .setStructValue(Struct.newBuilder()
            .putFields("$eq", Value.newBuilder()
                .setStringValue("action")
                .build()))
        .build())
    .build();

// Create new metadata to apply
Struct newMetadata = Struct.newBuilder()
    .putFields("updated", Value.newBuilder().setStringValue("true").build())
    .putFields("year", Value.newBuilder().setStringValue("2024").build())
    .build();

// Update vectors matching the filter
UpdateResponse response = index.updateByMetadata(filter, newMetadata, "example-namespace", false);
assertNotNull(response);
assertTrue(response.getMatchedRecords() > 0);

Dry Run Mode: Preview how many records would be updated without actually applying changes:

// Dry run to check how many records would be updated
UpdateResponse dryRunResponse = index.updateByMetadata(filter, newMetadata, "example-namespace", true);
assertNotNull(dryRunResponse);
int matchedRecords = dryRunResponse.getMatchedRecords();
assertTrue(matchedRecords > 0);

// Actually perform the update
UpdateResponse updateResponse = index.updateByMetadata(filter, newMetadata, "example-namespace", false);
assertNotNull(updateResponse);

Async Support
Both methods are available for AsyncIndex:

import com.google.common.util.concurrent.ListenableFuture;

// Fetch by metadata asynchronously
ListenableFuture<FetchByMetadataResponse> fetchFuture = 
    asyncIndex.fetchByMetadata("example-namespace", filter, 10, null);
FetchByMetadataResponse fetchResponse = fetchFuture.get();
assertNotNull(fetchResponse);

// Update by metadata asynchronously
ListenableFuture<UpdateResponse> updateFuture = 
    asyncIndex.updateByMetadata(filter, newMetadata, "example-namespace", false);
UpdateResponse updateResponse = updateFuture.get();
assertNotNull(updateResponse);
assertTrue(updateResponse.getMatchedRecords() > 0);

Note: These operations are supported for serverless indexes.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Infrastructure change (CI configs, etc)
  • Non-code change (docs, etc)
  • None of the above: (explain here)

Test Plan

Added integration tests.

@rohanshah18 rohanshah18 marked this pull request as ready for review November 12, 2025 22:12
@rohanshah18 rohanshah18 changed the base branch from main to rshah/release-candidate/2025-10 November 12, 2025 22:16
@rohanshah18 rohanshah18 changed the title Rshah/update fetch metadata Add fetch and update by metadata Nov 12, 2025
@rohanshah18 rohanshah18 merged commit ba32eba into rshah/release-candidate/2025-10 Nov 12, 2025
12 checks passed
@rohanshah18 rohanshah18 deleted the rshah/update-fetch-metadata branch November 12, 2025 22:42
rohanshah18 added a commit that referenced this pull request Nov 12, 2025
## Problem

Add fetch and update by metadata

## Solution

This PR adds the fetchByMetadata and updateByMetadata methods for both
Index and AsyncIndex clients, enabling fetching and updating vectors by
metadata filters.

**Fetch By Metadata**
Basic Fetch: Fetch vectors matching a metadata filter with optional
limit and pagination:
```java
import io.pinecone.proto.FetchByMetadataResponse;
import com.google.protobuf.Struct;
import com.google.protobuf.Value;

// Create a metadata filter
Struct filter = Struct.newBuilder()
    .putFields("genre", Value.newBuilder()
        .setStructValue(Struct.newBuilder()
            .putFields("$eq", Value.newBuilder()
                .setStringValue("action")
                .build()))
        .build())
    .build();

// Fetch vectors by metadata with limit
FetchByMetadataResponse response = index.fetchByMetadata("example-namespace", filter, 10, null);
assertNotNull(response);
assertTrue(response.getVectorsCount() > 0);

// Access fetched vectors
response.getVectorsMap().forEach((id, vector) -> {
    assertNotNull(vector);
    assertTrue(vector.hasMetadata());
});
```

Pagination Support: Use pagination tokens to fetch additional pages of
results:
```java
// Fetch first page
FetchByMetadataResponse firstPage = index.fetchByMetadata("example-namespace", filter, 2, null);
assertNotNull(firstPage);

// Fetch next page using pagination token
if (firstPage.hasPagination() && 
    firstPage.getPagination().getNext() != null && 
    !firstPage.getPagination().getNext().isEmpty()) {
    
    FetchByMetadataResponse secondPage = index.fetchByMetadata(
        "example-namespace", filter, 2, firstPage.getPagination().getNext());
    assertNotNull(secondPage);
}
```

**Update By Metadata**
Basic Update: Update vectors matching a metadata filter with new
metadata:
```java
import io.pinecone.proto.UpdateResponse;
import com.google.protobuf.Struct;

// Create a filter to match vectors
Struct filter = Struct.newBuilder()
    .putFields("genre", Value.newBuilder()
        .setStructValue(Struct.newBuilder()
            .putFields("$eq", Value.newBuilder()
                .setStringValue("action")
                .build()))
        .build())
    .build();

// Create new metadata to apply
Struct newMetadata = Struct.newBuilder()
    .putFields("updated", Value.newBuilder().setStringValue("true").build())
    .putFields("year", Value.newBuilder().setStringValue("2024").build())
    .build();

// Update vectors matching the filter
UpdateResponse response = index.updateByMetadata(filter, newMetadata, "example-namespace", false);
assertNotNull(response);
assertTrue(response.getMatchedRecords() > 0);
```

Dry Run Mode: Preview how many records would be updated without actually
applying changes:
```java
// Dry run to check how many records would be updated
UpdateResponse dryRunResponse = index.updateByMetadata(filter, newMetadata, "example-namespace", true);
assertNotNull(dryRunResponse);
int matchedRecords = dryRunResponse.getMatchedRecords();
assertTrue(matchedRecords > 0);

// Actually perform the update
UpdateResponse updateResponse = index.updateByMetadata(filter, newMetadata, "example-namespace", false);
assertNotNull(updateResponse);
```

**Async Support**
Both methods are available for AsyncIndex:
```java
import com.google.common.util.concurrent.ListenableFuture;

// Fetch by metadata asynchronously
ListenableFuture<FetchByMetadataResponse> fetchFuture = 
    asyncIndex.fetchByMetadata("example-namespace", filter, 10, null);
FetchByMetadataResponse fetchResponse = fetchFuture.get();
assertNotNull(fetchResponse);

// Update by metadata asynchronously
ListenableFuture<UpdateResponse> updateFuture = 
    asyncIndex.updateByMetadata(filter, newMetadata, "example-namespace", false);
UpdateResponse updateResponse = updateFuture.get();
assertNotNull(updateResponse);
assertTrue(updateResponse.getMatchedRecords() > 0);
```

Note: These operations are supported for serverless indexes.

## Type of Change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] This change requires a documentation update
- [ ] Infrastructure change (CI configs, etc)
- [ ] Non-code change (docs, etc)
- [ ] None of the above: (explain here)

## Test Plan

Added integration tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants