How to prevent Haystack from prefixing URL for transformers in model path?

[asharm0662]

Hi,

I am trying to create my own docker package for haystack/huggingface models.

I have downloaded the model I want to use locally, and can point to the directory for the model on my local machine no problem for my reader. This is the code I am using on my local machine to point the reader transformer reader tokenizer, to local directory of the model:

from haystack.nodes import TransformersReader, FARMReader
import json
from haystack.pipelines import ExtractiveQAPipeline, GenerativeQAPipeline
from haystack.nodes import TfidfRetriever
from haystack.document_stores import InMemoryDocumentStore
from haystack.utils import print_answers


document_store = InMemoryDocumentStore()

with open('data.txt') as jsonfile:
    dicts = json.load(jsonfile)
   
document_store.write_documents(dicts, duplicate_documents='overwrite')
# An in-memory TfidfRetriever based on Pandas dataframes
retriever = TfidfRetriever(document_store=document_store)
reader = TransformersReader(tokenizer="/path/to/model/distilbert-base-uncased", use_gpu=-1)
pipe = ExtractiveQAPipeline(reader, retriever)

prediction = pipe.run(
    query="Who is the father of Arya Stark?", params={"Retriever": {"top_k": 10}, "Reader": {"top_k": 2}}
    )
    
print_answers(prediction, details="minimal")

However when I wrap this code for Docker/serverless and point this line to the folder path where my model is:

reader = TransformersReader(tokenizer="/var/task/distilbert-base-uncased", use_gpu=-1)

Haystack is prefixing the URL like so:

reader = TransformersReader(tokenizer="https://huggingingface/var/task/distilbert-base-uncased", use_gpu=-1)

When I change the same line with model_path attribute, I can pick up the model locally but I am not getting the right answer:

reader = TransformersReader(model_name_or_path=/var/task/distilbert-base-uncased, tokenizer="/var/task/distilbert-base-uncased", use_gpu=-1)

How do I prevent this behavior? Let me know what or if any files I can provide.

[ZanSara]

Hello @asharm0662, actually the issue you're observing comes straight from the transformer's library, which takes care of handling the models. I don't think we can help too much with it. I'd recommend:

1. Checking the paths are correct
2. Checking file permissions
3. Have a look at the logs generated at startup and try to spot warnings from transformers (probably Haystack is not even noticing that there are issues finding the model, transformers seems to be handling it internally).

If you share the logs of you application I might be able to give you more hints.