Scaling MindsDB for High-Volume Workloads

[svencowart]

Hey MindsDB Team and Community,

My team is currently working on setting up MindsDB as the backbone of our machine learning platform, and while I love how it integrates ML into the database environment, I’m concerned about scaling MindsDB. Our platform will need to handle large-scale, real-time data pipelines with a high volume of concurrent model requests (thousands of requests per second). I’ve read Issue #2369 on GitHub, and the lack of a documentation regarding a scalable deployment model raises concerns about MindsDB’s performance at scale, especially without a straightforward way to horizontally scale.

To make sure we’re setting things up right and getting the performance we need, I’d love to get thoughts on a few questions:

1. Is there a good way to scale MindsDB horizontally? Can I safely say any advice here also covers horizontal scaling of lightwood?
2. Any tips for managing and serving a lot of models in real-time?

Thanks so much in advance for any guidance!

[marbemac]

Also very interested in this topic! Even some basic guidelines re what performance heuristics can be expected when a single node is deployed on xyz cpu/mem instance.

@svencowart where did you guys land? Did you end up deploying mindsdb to prod, and if so how has it gone?