feat: tests and example of custom embedding function

[Icemap]

No description provided.

[Mini256]

I couldn’t find a parameter to control the batch size in current code.

[Icemap]

It means you can input like 5 items into get_source_embeddings() function in one go. It gives you better throughput but higher memory usage.

[Mini256]

This section can be removed as we can go to pytidb's documentation to explain how to go about defining a custom function step by step.

[Icemap]

Should we? If a user goes here directly, at least they will have an overview of how many functions they should overwrite to make an embedding function class.

[Mini256]

Encounter an error:

Traceback (most recent call last):
File "/Users/xxxx/Projects/pytidb/examples/custom_embedding_function/main.py", line 25, in
embed_func = BGEM3EmbeddingFunction()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xxxx/Projects/pytidb/examples/custom_embedding_function/custom_embedding.py", line 46, in init
self._init_model()
File "/Users/xxxx/Projects/pytidb/examples/custom_embedding_function/custom_embedding.py", line 68, in _init_model
actual_dims = test_output["dense_vecs"].shape[1]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: tuple index out of range

[Icemap]

actual_dims = test_output["dense_vecs"].shape[1]

Removed the dimensions' judge logic due to the BGE-M3 only has one fixed dimension, which is 1024.

[Icemap]

And another thing is, it seems like we ran out of Jina AI's token:

ERROR tests/test_auto_embedding_image.py::test_image_search_with_query_text - litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Jina_aiException - litellm.Timeout: Connection timed out after None seconds.
ERROR tests/test_auto_embedding_image.py::test_image_search_with_image_path - litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Jina_aiException - litellm.Timeout: Connection timed out after None seconds.
ERROR tests/test_auto_embedding_image.py::test_image_search_with_pil_image - litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Jina_aiException - litellm.Timeout: Connection timed out after None seconds.

[Mini256]

Please ensure you have upgraded to pytidb==0.0.11, because in the new version, Text has been replaced with TEXT.

[Icemap]

Let me delete the venv environment and reinstall it again.

[Icemap]

Changed:

1. Text to TEXT
2. create_table(..., mode='overwrite') to create_table(..., mode='overwrite')

And ran main.py successfully.

[Mini256]

LGTM