Embedding failed with large-nli-stsb model but work for base model

[predoctech]

Trying to make use of the pre-trained model in sentence-transformer. It works for base model (tried bert-base and roberta-base) but failed in large models (for roberta and I think bert as well) with the following errors at time of embedding corpus:

Seems like size of the embedding is too large to write as Elasticsearch index?

[tanaysoni]

Hi @predoctech, this might be due to the indexing call to Elasticsearch timing out. The timeout is increased from default 10 seconds to 30 seconds in #119. Can you pull the latest master and try again to see if it resolves the issue?

[predoctech]

Hi @tanaysoni, when I tested the changes today the timeout error still persists. Is a 30 second timeframe still not enough, or there is something else going on with indexing call?

[tanaysoni]

Hi @predoctech, how large are the documents(number of characters or bytes) you're indexing? Can you try if the indexing works for a single document instead of a batch?

[predoctech]

@tanaysoni , my document are FAQ pairs of 970 rows all in a single document. That isn't too large a document isn't it, and the base models worked just the large model timed out.

[tanaysoni]

Hi @predoctech, yes, you're right it might be possible that 30 seconds is not enough. Can you try again with a very large timeout(eg, 600) here?

[predoctech]

Hi @tanaysoni , I tried your suggestion and now the error is not with timeout but with the embedding itself. Specifically I noticed the following error message:
"BulkIndexError: ('500 document(s) failed to index.', [{'create': {'_index': 'document2', '_type': '_doc', '_id': 'riFKUHIB1cG6QRRa1aQo', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': 'failed to parse', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Field [question_emb] of type [dense_vector] of doc [riFKUHIB1cG6QRRa1aQo] has exceeded the number of dimensions [768] defined in mapping'}..."
Not sure why my data now exceeded the vector dimension defined for the DocumentStore?
N.B. I'm trying to load roberta-large-nli-stsb-mean-tokens

[tholor]

The larger models like roberta-large-nli-stsb-mean-tokens create embeddings of dimensionality 1024. Should work fine if you set embedding_dim=1024 when initializing your Documentstore.

[predoctech]

Thanks @tholor . That's right, setting 1024 as dimensionality for large models does work.
@tanaysoni now with setting a longer timeout the issue of indexing is resolved too. I used timeout=300, which seems to be sufficient for all large models I have tried.
Trust that change would be merge into code base? Thanks both.

[tholor]

Fixed in #130