Generating exe with langchain-module not possible?

[Rapid1898-code]

Hello - i try to create an executeable for windows with the following code - the program works fine when i run it with python:

from langchain.document_loaders import DirectoryLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.llms import CTransformers
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.prompts import PromptTemplate 
from langchain.chains import RetrievalQA
import os
import sys
import timeit

if __name__ == '__main__':
  print(f"Program name: {os.path.basename(__file__)}")
  path = os.path.abspath(os.path.dirname(sys.argv[0]))   

  ### Interpret content of local files
  print(f"Prepare to interpret the content of local files and store to a vector-db...")
  start = timeit.default_timer()
  text_loader_kwargs={'autodetect_encoding': True}
  loader = TextLoader("urlFull.txt", encoding="utf8")
  documents = loader.load()
  splitter = RecursiveCharacterTextSplitter(chunk_size=250, chunk_overlap=50)
  texts = splitter.split_documents(documents)
  embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2", model_kwargs={'device': 'cpu'})

  # create and save the local database
  db = FAISS.from_documents(texts, embeddings)
  db.save_local("robalf")
  stop = timeit.default_timer() 
  print(f"Runtime for this part: {round((stop-start)/60,2) }")

  ### prepare AI with local files
  print(f"Answer the question using Llama nlp...")
  start = timeit.default_timer()
  print(f"Prepare template...")
  # prepare the template we will use when prompting the AI

  template = """Use the following text to answer the user's question.
  If you don't know the answer, just say that you don't know, don't try to make up an answer.
  Context: {context}
  Question: {question}
  Only return the helpful answer in english below as a csv-file and nothing else.
  Helpful answer:
  """

  print(f"Load language model...")
  # load the language model
  try:
    fn = os.path.join(path, "llama-2-7b-chat.ggmlv3.q8_0.bin")
    llm = CTransformers(model=fn,
                        model_type='llama',
                        config={'max_new_tokens': 1024, 'temperature': 0.01})
  except:  
    print(f"Llama package <llama-2-7b-chat.ggmlv3.q8_0.bin> not found in program path...")

  try:
    fn = os.path.join("G:\DEV\TEMPDEV", "llama-2-7b-chat.ggmlv3.q8_0.bin")
    llm = CTransformers(model=fn,
                        model_type='llama',
                        config={'max_new_tokens': 1024, 'temperature': 0.01})
  except:
    print(f"Llama package <llama-2-7b-chat.ggmlv3.q8_0.bin> also not found in other path...")    

  print(f"Load the interpreted information from the local database...")
  # load the interpreted information from the local database
  embeddings = HuggingFaceEmbeddings(
      model_name="sentence-transformers/all-MiniLM-L6-v2",
      model_kwargs={'device': 'cpu'})
  db = FAISS.load_local("robalf", embeddings)

  print(f"Prepare a version of the llm pre-loaded with the local content...")
  # prepare a version of the llm pre-loaded with the local content
  retriever = db.as_retriever(search_kwargs={'k': 2})
  prompt = PromptTemplate(
      template=template,
      input_variables=['context', 'question'])

  qa_llm = RetrievalQA.from_chain_type(llm=llm,
                                      chain_type='stuff',
                                      retriever=retriever,
                                      return_source_documents=True,
                                      chain_type_kwargs={'prompt': prompt})

  # ask the AI chat about information in our local files
separate them with a comma. "
  # prompt = "Create the output as a comma seperated csv-file. List a maximum of 20 products produced or sold by the company in the text below."
  prompt = "Listen Sie im folgenden Text maximal 20 Produkte auf, die vom Unternehmen hergestellt oder verkauft werden."

  output = qa_llm({'query': prompt})
  print(output["result"])
  stop = timeit.default_timer() 
  print(f"Runtime for this part: {round((stop-start)/60,2) }")

I create the executeable using this command:
pyinstaller --onefile --collect-data frozendict urlAnswering.py

When i run the executeable i get this error-message:

G:\DEV\Python-Diverses\Langchain>urlAnswering.exe
Traceback (most recent call last):
  File "G:\DEV\Python-Diverses\Langchain\URLAnswering.py", line 13, in <module>
    from langchain.chains import RetrievalQA
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "langchain\chains\__init__.py", line 50, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "langchain\chains\llm_summarization_checker\base.py", line 19, in <module>
  File "langchain\prompts\prompt.py", line 184, in from_file
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\marku\\AppData\\Local\\Temp\\_MEI14882\\langchain\\chains\\llm_summarization_checker\\prompts\\create_facts.txt'
[21872] Failed to execute script 'URLAnswering' due to unhandled exception!

Why is this not working?

[rokm]

FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\marku\AppData\Local\Temp\_MEI14882\langchain\chains\llm_summarization_checker\prompts\create_facts.txt'

Looks like you need to collect data from langchain as well (--collect-data langchain).

[Rapid1898-code]

thx - i tried it now witht his command
pyinstaller --onefile --collect-data langchain urlAnswering.py

But when i run the executable i get this error now:

G:\DEV\Python-Diverses\Langchain>urlAnswering.exe
Program name: URLAnswering.py
Prepare to interpret the content of local files and store to a vector-db...
Traceback (most recent call last):
  File "transformers\utils\versions.py", line 102, in require_version
  File "importlib\metadata\__init__.py", line 984, in version
  File "importlib\metadata\__init__.py", line 957, in distribution
  File "importlib\metadata\__init__.py", line 548, in from_name
importlib.metadata.PackageNotFoundError: No package metadata was found for tqdm

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "langchain\embeddings\huggingface.py", line 58, in __init__
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "sentence_transformers\__init__.py", line 3, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "sentence_transformers\datasets\__init__.py", line 3, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "sentence_transformers\datasets\ParallelSentencesDataset.py", line 4, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "sentence_transformers\SentenceTransformer.py", line 11, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "transformers\__init__.py", line 26, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 391, in exec_module
  File "transformers\dependency_versions_check.py", line 57, in <module>
  File "transformers\utils\versions.py", line 117, in require_version_core
  File "transformers\utils\versions.py", line 104, in require_version
importlib.metadata.PackageNotFoundError: No package metadata was found for The 'tqdm>=4.27' distribution was not found and is required by this application.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "G:\DEV\Python-Diverses\Langchain\URLAnswering.py", line 33, in <module>
    embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2", model_kwargs={'device': 'cpu'})
  File "langchain\embeddings\huggingface.py", line 61, in __init__
ImportError: Could not import sentence_transformers python package. Please install it with `pip install sentence-transformers`.
[7056] Failed to execute script 'URLAnswering' due to unhandled exception!

[bwoodsend]

If the error message asks for package metadata, you need to give it package metadata. Try adding --copy-metadata=tqdm.

[Rapid1898-code]

Thx - but not the exe-file hast over 350MB - is this possible to shinken this a little bit?

[rokm]

Thx - but not the exe-file hast over 350MB - is this possible to shinken this a little bit?

sentence_transformers sounds like something that uses a deep learning framework (torch?), so the answer is likely "no".

[rokm]

But I guess you can always look what the largest files in the frozen application are, and see if you can do without them...

[Rapid1898-code]

I was still not able to bring this to running - allways another package is missing.
If anybody has allready created an executable using langchain - pls provide me your detailed solutions.