Add File Retrieval to Agents

[tmichaeldb]

Requires https://github.com/mindsdb/mindsdb/pull/9098/files for testing.

This PR adds file retrieval to agents through the retrieval skill.

Example usage:

agent = agents.get('my_agent')
agent.add_file('./hooblyblob.txt', 'It has information about the company hooblyblob')
agent.completion([{'question': 'When was hooblyblob founded?', 'answer': None}])

with contents of ./hooblyblob.txt:

Hooblyblob was founded in 2024.

Steps taken through the SDK:

1. Check if the file exists from filename or upload to MindsDB if it doesn't exist
2. Insert the file contents into a knowledge base (can optionally pass existing KB, otherwise a new one is created for the file)
3. Create a new retrieval skill using the knowledge base from 2
4. Add the new retrieval skill to the agent

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
mindsdb_sdk
agents.py	122	45	63%	20, 79, 87, 90, 94, 96, 98, 100, 102, 168–206, 229–231, 252–254, 258–262
databases.py	42	2	95%	107, 134
handlers.py	39	1	97%	77
jobs.py	65	3	95%	96–99
knowledge_bases.py	110	12	89%	56–59, 139, 143–152, 230–233
ml_engines.py	42	3	93%	94, 126, 128
models.py	181	13	93%	107, 195, 204, 206, 276, 308, 336, 457, 465, 484, 500, 527, 531
projects.py	63	1	98%	167
query.py	13	1	92%	14
skills.py	50	7	86%	43, 45, 49, 56, 72–76, 118
tables.py	108	4	96%	177, 189, 205, 297
views.py	37	2	95%	105, 138
mindsdb_sdk/connectors
rest_api.py	172	28	84%	16–26, 32–33, 48, 69–71, 88, 91, 98–101, 152–160
TOTAL	1116	122	89%

Tests	Skipped	Failures	Errors	Time
14	0 💤	0 ❌	0 🔥	8.039s ⏱️

[dusvyat]

other than minor points, looks good 🚀

[dusvyat]

We should integrate chunking here. Can be alright for now but def one for after hackathon if we don't have time

[tmichaeldb]

I was imagining we would handle chunking on the KB mindsdb side. When doing a KB insert we could simply check if we're selecting from the files table, then do chunking before inserting into the VectorDB.

So yes, we should integrate chunking in some way for sure.

[tmichaeldb]

I updated this PR to use the knowledge_base skill type instead of retrieval for now while we polish retrieval to be working 100%.