R&D: Structured Data

[gmpetrov]

Semantic search does not perform well with structured data (e.g: CSV file)

I think it does not make sense to store this kind of data in a vector database.

Should we create an SQL table on the fly?

How to make it work seamlessly for the end user? Can it work in combination with Datastores or should we treat it as another tool?

Solution requirements

• At runtime, the full CSV or SQL table should be loaded in memory

Ressources:

• https://js.langchain.com/docs/modules/chains/other_chains/sql

[pax-k]

If you want to answer questions like "how many sales did I have in May", then SqlDatabaseChain should work as you suggested https://js.langchain.com/docs/use_cases/tabular/

But if you don't care about querying your data and just want to do semantic search, eg "do you know something about X", I'm thinking that CSVLoader shouldn't be discarded. Did you have bad experience with it?

[BenoitGdb]

Check this out : https://openai.com/blog/function-calling-and-other-api-updates