feat(retrievers)!: Move all retriever config options to RetrieverOptions

[davidmigloz]

Continuing the efforts to add full LCEL support to all core components.

In this case, the Retriever class didn't allow to pass any options when invoking it. VectorStoreRetriever only supported config options in the constructor, which forced users to re-create the retriever every time they wanted to change the options.

A typical use case that was hard to implement was to do an initial search with a high scoreThreshold and then repeat the search with a lower scoreThreshold if the first search didn't return any results.

Now the Retriever classes support defining default options when instantiating them, but you can always override them when invoking the retriever.

const query = 'How much does my order cost?';
// Default options
final retriever = vectorStore.asRetriever(
  defaultOptions: const VectorStoreRetrieverOptions(
    searchType: VectorStoreSimilaritySearch(scoreThreshold: 0.8),
  ),
);

// We invoke the retriever with the default options
List<Document> res = await retriever.invoke(query);

// If empty, we override the default options with a lower score threshold
if(res.isEmpty) {
  res = await retriever.invoke(
    query,
    options: const VectorStoreRetrieverOptions(
      searchType: VectorStoreSimilaritySearch(scoreThreshold: 0.5),
    ),
  );
}

You can also change the options in a Runnable pipeline using the bind method.

final retriever = vectorStore.asRetriever();
final chain = Runnable.fromMap<String>({
  'context1': retriever.bind(retrieverOptions1),
  'context2': retriever.bind(retrieverOptions2),
  'question': Runnable.passthrough(),
}).pipe(promptTemplate).pipe(chatModel).pipe(outputParser);

Migration

Getting a retriever from a vector store

The asRetriever method now takes an optional defaultOptions parameter instead of a searchType.

Before:

final retriever = vectorStore.asRetriever(
  searchType: const VectorStoreSimilaritySearch(k: 10),
);

Now:

final retriever = vectorStore.asRetriever(
  defaultOptions: const VectorStoreRetrieverOptions(
    searchType: VectorStoreSimilaritySearch(k: 10),
  ),
);

Custom retrievers

If you have any custom retrievers, the base retriever class has been renamed from BaseRetriever to Retriever.
You can also define a custom retriever options class (that extends from RetrieverOptions) if you retriever supports additional configuration options.

Before:

class MyCustomRetriever extends BaseRetriever {
  // ...

  @override
  Future<List<Document>> getRelevantDocuments(final String query) {
    // ...
  }
}

Now:

class MyCustomRetriever Retriever<MyCustomRetrieverOptions> {
  // ...

  @override
  Future<List<Document>> getRelevantDocuments(
    final String query, {
    final MyCustomRetrieverOptions? options,
  }) {
    // ...
  }
}