[Bug]: Can't import litellm with latest release

[davidmezzetti]

What happened?

Can't import litellm with latest release

Relevant log output

>>> import litellm

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/test/lib/python3.8/site-packages/litellm/__init__.py", line 590, in <module>
    from .llms.bedrock import (
  File "/tmp/test/lib/python3.8/site-packages/litellm/llms/bedrock.py", line 285, in <module>
    class AmazonMistralConfig:
  File "/tmp/test/lib/python3.8/site-packages/litellm/llms/bedrock.py", line 301, in AmazonMistralConfig
    stop: Optional[list[str]] = None
TypeError: 'type' object is not subscriptable

Twitter / LinkedIn details

No response

[davidmezzetti]

Related: neuml/txtai#682

[davidmezzetti]

@krrishdholakia FYI - Looks like the latest release on PyPI is broken, at least for Python 3.8.

[ishaan-jaff]

Caused by this PR: #2290
Sorry for the issue, unfortunately our AWS account was suspended and we could not test this on our ci/cd

Will fix this by EOD today @davidmezzetti + we just got a new AWS account so our ci/cd should be back up soon

[ishaan-jaff]

Really sorry about this @davidmezzetti, do you have any suggestions / feedback on anything we can do to prevent this error you're seeing? As you pointed out it's the 2nd time you've seen an error like this and I don't want there to be a 3rd

[davidmezzetti]

Just noted this here: neuml/txtai#682 (comment)

is this now fixed? @ishaan-jaff @davidmezzetti

[davidmezzetti]

@krrishdholakia

Unfortunately with v1.29.1, I get the following error with Python 3.8.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/test/lib/python3.8/site-packages/litellm/__init__.py", line 574, in <module>
    from .llms.huggingface_restapi import HuggingfaceConfig
  File "/tmp/test/lib/python3.8/site-packages/litellm/llms/huggingface_restapi.py", line 12, in <module>
    from .prompt_templates.factory import prompt_factory, custom_prompt
  File "/tmp/test/lib/python3.8/site-packages/litellm/llms/prompt_templates/factory.py", line 588, in <module>
    def extract_between_tags(tag: str, string: str, strip: bool = False) -> list[str]:
TypeError: 'type' object is not subscriptable

[ishaan-jaff]

Fix here: #2344 @davidmezzetti

[ishaan-jaff]

Fixed on here: https://github.com/BerriAI/litellm/releases/tag/v1.29.5

@davidmezzetti can you lmk if this works for you, if confirmed going to add testing for this case to ensure it does not occur again

[ishaan-jaff]

leaving open until it's confirmed this is fixed + we've added testing for python 3.8

[davidmezzetti]

@ishaan-jaff @krrishdholakia

There is a similar but different error with the latest version as of right now (v1.30.2)

>>> import litellm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/test/lib/python3.8/site-packages/litellm/__init__.py", line 575, in <module>
    from .llms.huggingface_restapi import HuggingfaceConfig
  File "/tmp/test/lib/python3.8/site-packages/litellm/llms/huggingface_restapi.py", line 12, in <module>
    from .prompt_templates.factory import prompt_factory, custom_prompt
  File "/tmp/test/lib/python3.8/site-packages/litellm/llms/prompt_templates/factory.py", line 631, in <module>
    def extract_between_tags(tag: str, string: str, strip: bool = False) -> list[str]:
TypeError: 'type' object is not subscriptable

If a version of the tests run on Python 3.8, I would assume that this error would be raised anytime it's reintroduced. It seems like developers are only using Python 3.9+.

[davidmezzetti]

Pinging again on this @ishaan-jaff @krrishdholakia

I'm continually getting asked about this. It's a really bad error as simply installing litellm breaks my entire project as importing the library is a breaking action.

If Python 3.8 isn't a priority, please also let me know that. I may have to look at an alternative because this affects my project's reputation.