[FEATURE REQUEST] Pls. add Hugging Face inference as option

[vanetreg]

For us, who use Windows machines without decent HW / GPU, so using better local LLMs via Ollama is not a really good option, a good option would be using eg. Huggin Face's inference, so - as I see CrewAI uses Langchain - it should not be an issue, since LCh has HF integration:
https://python.langchain.com/docs/integrations/platforms/huggingface
So pls. add this option ( so we should only add our HF API key ) and testing your code would not break our OpenAI budget :)

[joaomdmoura]

Hey there, I'll double check but giving we use langchain it should work out of the box, but I'll test and add some examples

[vanetreg]

OK, than I'll also test it.

[llxxxll]

@vanetreg @joaomdmoura
If you want to save some OpenAI budget, you can use a locally run Language Model (LLM) for testing. I use ChatGLM-6B and leverage the ChatOpenAI method provided by langchain.chat_models. Since the locally run LLM doesn't require an API key, you can use any string (I use 'xxx') as a placeholder. The specific code is as follows:

from langchain.chat_models import ChatOpenAI

local_llm = ChatOpenAI(
  base_url="http://192.168.1.3:8000/v1",
  api_key = "xxx"
)

researcher = Agent(
  role='Researcher',
  goal='Discover new insights',
  backstory="",
  verbose=True,
  allow_delegation=False,
  llm = local_llm
)

[joaomdmoura]

Great example, might be worth adding to the README together withe Ollama to provide more options for folks running things locally! Curious to hear more on how ChatGLM-6B is performing!

[llxxxll]

@joaomdmoura In some translation tasks, I use ChatGPT for preliminary translation, THUDM/ChatGLM-6B for proofreading, and then ChatGPT again for precise translation. For simpler and exploratory tasks, I initially use THUDM/ChatGLM-6B to streamline the process before incorporating ChatGPT, which significantly saves on development costs. However, it's worth noting that THUDM/ChatGLM-6B is specifically tailored for Chinese, and similar demands may arise for small models in other languages. Nevertheless, the trend for LLMs is a gradual reduction in costs, and this approach may be a temporary measure.