Skip to content

Conversation

@bevzzz
Copy link
Collaborator

@bevzzz bevzzz commented Oct 17, 2025

Here's an example showcasing the new functionality.

client.collections.create(
  "Songs", c -> c
    .generativeModule(Generative.anthropic(a -> a
      .mode("claude-sonnet")
      .maxTokens(2048)
      .temperature(.3f)
      .stopSequences("\n\n", "{end}")
);

var songs = client.collections.use("Songs");

var dissTracks = songs.generate.nearVector(
    myFavouriteSong,
    nv -> nv.distance(0.42f).limit(25),
    generate -> generate.singlePrompt(
      "write a rap diss based on the lyrics",
      rag -> rag.dynamicProvider(DynamicProvider.xai(
        x -> x.model("grok-1").topP(.71f)
      ))
    ),
    GroupBy.property("artist", 5, 5)
);

for (var group : dissTracks.groups().values()) {
   var diss = group.generated().text();
   System.out.printf("Diss track for group %s: %s\n", group.name(), diss);
}

This PR provides support for all generative modules we currently have in Weaviate:

  • Anyscale
  • Anthropic
  • Aws
  • Cohere
  • Databricks
  • Friendliai
  • Google
  • Mistral
  • Nvidia
  • Ollama
  • OpenAI / Azure
  • Xai

DummyGenerative, while defined, is not "advertised" via the static factories in Generative and DynamicProvider classes.

Copy link

@orca-security-eu orca-security-eu bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

@bevzzz bevzzz changed the title v6: Generative queries v6: Generative queries + Dynamic RAG Oct 22, 2025
@bevzzz bevzzz marked this pull request as ready for review October 22, 2025 14:27
*
* @param fn Lambda expression for optional parameters.
*/
public static DynamicProvider aws(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DId you think about using separate factories for sagemaker and bedrock?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did but after looking through the docs ("Ask AI") I figured it was only 1 argument each that is special to the service (sagemaker -> endpoint, bedrock -> model).

Providing separate builders for each is not too much work, but I'm leaning towards documenting that in the Javadoc. Wdyt?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is not much work, I'd prefer the separate builders

But no strong opinion

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this change in a follow-up PR, to avoid tricky rebases and such.

If everything else is fine, I'll proceed with merging this.

@bevzzz bevzzz merged commit 03d711f into v6 Oct 27, 2025
2 checks passed
@bevzzz bevzzz deleted the v6-generate branch October 27, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants