Skip to content
This repository has been archived by the owner on Oct 30, 2024. It is now read-only.

add: cohere_rerank postprocessor + expand env vars in flows config #31

Merged
merged 2 commits into from
Jun 24, 2024

Conversation

iwilltry42
Copy link
Collaborator

@iwilltry42 iwilltry42 commented Jun 20, 2024

This PR adds the cohere_rerank postprocessor.
Cohere offers free trial keys: https://dashboard.cohere.com/api-keys

Additionally, environment variables ${var} and $var are now expanded in the flows-config files, so ${COHERE_API_KEY} in the example config below is inferred from the environment.

flows:
  foo:
    default: false
    ingestion:
      - filetypes: [".md"]
        documentloader:
          name: plaintext
        textsplitter:
          name: markdown
        transformers:
          - name: filter_markdown_docs_no_content
    retrieval:
      retriever:
        name: basic
        options:
          topK: 25
      postprocessors:
        - name: cohere_rerank
          options:
            apiKey: ${COHERE_API_KEY}
            topN: 3
            model: rerank-multilingual-v3.0

@iwilltry42 iwilltry42 merged commit d0fe45e into main Jun 24, 2024
1 check passed
@iwilltry42 iwilltry42 changed the title add: cohere_rerank postprocessor add: cohere_rerank postprocessor + expand env vars in flows config Jun 24, 2024
@iwilltry42 iwilltry42 deleted the feat/reranking branch June 24, 2024 13:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant