Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AnthropicLLM #444

Merged
merged 18 commits into from
Mar 23, 2024
Merged

Add AnthropicLLM #444

merged 18 commits into from
Mar 23, 2024

Conversation

sdiazlor
Copy link
Contributor

@sdiazlor sdiazlor commented Mar 19, 2024

Closes #424

from distilabel.llm.anthropic import AnthropicLLM

if __name__ == "__main__":
    llm = AnthropicLLM(model="claude-3-opus-20240229", api_key="api.key")
    llm.load()
    output = llm.generate(
        inputs=[
            [
                {
                    "role": "user",
                    "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
                },
            ]
        ]
    )
    print(output)

@sdiazlor sdiazlor self-assigned this Mar 19, 2024
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
@davidberenstein1957 davidberenstein1957 added this to the 1.0.0 milestone Mar 19, 2024
@davidberenstein1957 davidberenstein1957 linked an issue Mar 19, 2024 that may be closed by this pull request
Copy link
Member

@alvarobartt alvarobartt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sdiazlor! Thanks for the PR, there's a couple of things I've mentioned and some others I'd like to confirm as:

  • Is there any way we can check whether the provided model is an existing one? Could we offer a listing or just call an API endpoint to get the list of existing models within Anthropic?
  • Could you reply with an example using the http_client? Or just mentioning which are the scenarios where it's useful? Also note that there may be issues when serializing the http_client if that appears to expect an httpx.Client instance

pyproject.toml Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
tests/unit/llm/test_anthropic.py Outdated Show resolved Hide resolved
sdiazlor and others added 4 commits March 20, 2024 09:23
Co-authored-by: Alvaro Bartolome <alvaro@argilla.io>
Co-authored-by: Alvaro Bartolome <alvaro@argilla.io>
Co-authored-by: Alvaro Bartolome <alvaro@argilla.io>
@sdiazlor
Copy link
Contributor Author

sdiazlor commented Mar 20, 2024

  • Is there any way we can check whether the provided model is an existing one? Could we offer a listing or just call an API endpoint to get the list of existing models within Anthropic?

@alvarobartt Thanks for your feedback.

1- We could do that by using something similar to this. Should we implement it?

from anthropic import AsyncAnthropic
def _check_model_exists(self) -> None:
    """Checks if the specified model exists in the available models."""
    annotation = get_type_hints(AsyncAnthropic().messages.create).get('model', None)
    models= [value for type_ in get_args(annotation) if get_origin(type_) is Literal for value in get_args(type_)]

    if self.model not in models:
        raise ValueError(f"Model {self.model} does not exist among available models.")

2- They use it to customize the httpx client as follows, so yeah, maybe this is an issue for serialization. Not sure how you handled this in similar cases, or if you just decided to remove it.

from anthropic import Anthropic

client = Anthropic(
    # Or use the `ANTHROPIC_BASE_URL` env var
    base_url="http://my.test.server.example.com:8083",
    http_client=httpx.Client(
        proxies="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)```

Copy link
Member

@gabrielmbmb gabrielmbmb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, it looks good!. Just need a small update before merging.

src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Outdated Show resolved Hide resolved
src/distilabel/llm/anthropic.py Show resolved Hide resolved
@gabrielmbmb gabrielmbmb merged commit 901fe7d into core-refactor Mar 23, 2024
4 checks passed
@gabrielmbmb gabrielmbmb deleted the add-claude-llm branch March 23, 2024 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add AnthropicLLM
4 participants