Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for CohereForAI/c4ai-command-r-v01 tool template #556

Merged
merged 5 commits into from
Mar 15, 2024

Conversation

xenova
Copy link
Contributor

@xenova xenova commented Mar 15, 2024

This PR adds the missing language features to support the new function calling template by Cohere, for their new https://huggingface.co/CohereForAI/c4ai-command-r-v01 model. This includes:

  • Tuple types
  • Iterator unpacking in for loops
  • object.items() built-in
  • | items and length filters for objects

cc @drbh for TGI e2e test case
cc @Rocketknight1 for transformers
cc @Wauplin for chat widget

@xenova xenova merged commit 73a7a56 into main Mar 15, 2024
2 checks passed
@xenova xenova deleted the jinja-more-features-4 branch March 15, 2024 01:10
@pcuenca
Copy link
Member

pcuenca commented Mar 15, 2024

Very cool!

Copy link
Member

@julien-c julien-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very cool, let's make this big when it's out

@xenova
Copy link
Contributor Author

xenova commented Mar 15, 2024

Now it's compatible with Transformers.js too! 😎 e.g.,

import { AutoTokenizer } from "@xenova/transformers";

const tokenizer = await AutoTokenizer.from_pretrained("Xenova/c4ai-command-r-v01-tokenizer")

// define conversation input:
const conversation = [
  { role: "user", content: "Whats the biggest penguin in the world?" }
]
// define documents to ground on:
const documents = [
  { title: "Tall penguins", text: "Emperor penguins are the tallest growing up to 122 cm in height." },
  { title: "Penguin habitats", text: "Emperor penguins only live in Antarctica." }
]

// render the RAG prompt as a string:
const grounded_generation_prompt = tokenizer.apply_chat_template(
  conversation,
  {
    chat_template: "rag",
    tokenize: false,
    add_generation_prompt: true,

    documents,
    citation_mode: "accurate", // or "fast"
  }
)
console.log(grounded_generation_prompt);

I'm ready to launch whenever. Might be good to align with @Rocketknight1 on this? If it will take too long, I can do a soft launch today too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants