0.9.2

Added

• joiner param to text_helper.text_split() for better control of regex separator handling
• query filter mix-in, embedding.pgvector.match_oneof(), for use with meta_filter argument to DB.search
• llm_wrapper.response_type to differentiate tool calling vs regular LLM responses

Changed

• Index word loom items by their literal default language text, as well
• Cleaned up PGVector query-building logic

Fixed

• llm_wrapper.llm_response objects to handle tool calls
• failure of some matching scenarios in embedding.pgvector.match_exact()

Removed

• Previously deprecated first_choice_text & first_choice_message methods

0.9.1

[0.9.1] - 20240604

Fixed

• Demos & testing around text_helper.text_split_fuzzy()

0.9.0

[0.9.0] - 20240604

Added

• text_helper.text_split_fuzzy() as basically a generator version of text_helper.text_splitter(), and deprecate the latter
• text_helper.text_split() which brooks no overlap
• embedding.pgvector_data module for vector database embedding of data fields using PGVector. Document fields are now treated as simple text, with specialized metadata generalized in the new metadata field
• query filter mix-in, embedding.pgvector.match_exact(), for filtering vector search by metadata fields via new meta_filter argument to DB.search. This restores some of the tag matching search functionality that's been removed.
• clone() method on wordloom.language_item class (formerly text_item)
• meta and preserve_key args on wordloom.language_item (formerly text_item) initializer, to preserve TOML table items and top-level TOML key for each language item, respectively, in object properties

Changed

• Deprecation of text_helper.text_splitter()
• tags field of table encapsulated in embedding.pgvector_data now modified into a general, JSON meta field
• Word Loom now uses _ as the TOML table key for text content. The former text is deprecated
• Word Loom now uses _m as the TOML table key for text substitution markers. The former markers is deprecated
• Word Loom now reserves TOML table keys beginning with _, and passes on all other keys to the new meta property
• wordloom.text_item class now renamed wordloom.language_item
• lang arg on wordloom.language_item (formerly text_item) initializer renamed deflang

Removed

• embedding.pgvector_data_doc (now just embedding.pgvector_data)
• conjunctive option for tags searching now removed, in favor of a query filter mix-in approach

0.8.0

[0.8.0]

Added

• llm_wrapper.llama_cpp_http_chat & llm_wrapper.llama_cpp_http; llama.cpp low-level HTTP API support
• llm_wrapper.llama_response class with flexible handling across API specs
• window init param for for embedding.pgvector.MessageDB, to limit message storage per history key

Changed

• Deprecated first_choice_text & first_choice_message methods in favor of first_choice_text attributes on response objects
• Clarify set quite setup docs

0.7.1

[0.7.1] - 2024-02-22

Added

• MessageDB.get_messages() options: since (for retrieving messages aftter a timestamp) and limit (for limiting the number of messages returned, selecting the most recent)

Changed

• PGVector users now manage their own connection pool by default
• Better modularization of embeddings test cases; using conftest.py more
• pgvector_message.py PG table timstamp column no longer a primary key

Fixed

• Backward threshold check for ogbujipt.embedding.pgvector_data_doc.DataDB

0.7.0

What's changed

Added

• Command line options for demo/chat_web_selects.py
• Helper for folks installing on Apple Silicon: constraints-apple-silicon.txt
• Function calling demo
• ogbujipt.embedding.pgvector_message.insert_many()

Changed

• Improved use of PGVector helper SQL query parameters
• PGVector helper search(query_tags=[..]) now uses contains operator (filters by existence in tag sets), not the same as where tags are OR
• PGVector helper search can now be set to work conjunctively or disjunctively
• PGVector helper query now has threshold arg based on degree of similarity. limit default now unlimited. Use SQL query args for query_embedding.
• embedding.pgvector split into a couple of modules.
• Separated data-style PGVector DBs from doc-style. tags is no longer the final param for PGVector docs helper methods & some params renamed.
• PGVector helper method results now as attr_dict
• PGVector helper now uses connection pooling & is more multiprocess safe
• ogbujipt.embedding.pgvector_chat renamed to ogbujipt.embedding.pgvector_message
• DB MIGRATION REQUIRED - ogbujipt.embedding.pgvector_message table schema

Fixed

• insert_many PGVector helper method; semantics & performance
• demo/chat_web_selects.py & demo/chat_pdf_streamlit_ui.py (formerly non-functional)
• Tests & CI for PGVector helper

0.6.2

What's changed

Changed

• OgbujiPT/embedding/pgvector.py
  • Message rows in the MessageDB changed to a TIMESTAMP WITH TIME ZONE PRIMARY KEY from a TIMESTAMP PRIMARY KEY

0.6.1

What's changed

Changed

• OgbujiPT/embedding/pgvector.py
  • Message rows in the MessageDB changed to a TIMESTAMP PRIMARY KEY from a SERIAL PRIMARY KEY

0.6.0

What's Changed

Added

• Support for efficient multi-queries (executemany): insert_many vs insert
• Chatlog-specific PGVector helper (PGvectorHelper specialized into DocDB & MessageDB)
• PG Vector DB instance launch fo ruse in test suite & GitHub actions
• Updated model styles and prompt formatting, particularly for improved closed-context patterns & per-context metadata (e.g. for Airboros)

Changed

• Model introspection moved to llm_wrapper classes: hosted_model & available_models
• Move OAI API response structure handling helpers to be static methods of the llm_wrapper classes
• Clarified demo names
• Support upstream python-openai > 1.0

Fixed

• README sample code
• Demos
• Test cases
• Use of string formatting intead of SQL query parameters
• Registration of vector type
• pgvector test case

Full Changelog: 0.5.1...0.6.0

0.5.1

Added functionality to the PGvectorConnection class to take no SentenceTransformer object, making it more useful for generic data storage