Add support for SQLite

[aslotte]

Registering models and metadata in the cloud is not always feasible.
To that effect we want to be able to add a provider to store these things on-premise

[aslotte]

We probably want to start off with supporting SQL Server and a local file share. It would make sense that this was added to a separate package MLOps.NET.Local.SqlServer

[dcostea]

@aslotte , do you mind if I'm considering adding SQLite? I think it will do just fine here, instead of SqlServer. But we can think of doing both, SqlServer and SQLite.

[aslotte]

@dcostea we can certainly look at SQLite if you like. My thinking is that most organizations that want to start of locally will have some kind of SQL server set up, but that may be a misconception :)
Would love to discuss it tomorrow if you're joining the stream. I'm thinking we may want to spend a bit more time finding a general API we're happy with in Azure before working on the local solution.

[dcostea]

@aslotte , sure, let's talk tomorrow.

[sammysemantics]

I agree with @dcostea. For development purposes, using SQLite is easier to use instead of a full server setup. I completely agree that enterprises will want to use connect with SQL server, but that can be implemented later.

Also, I'm unsure of the naming convention of MLOps.NET.Local._______. I am unsure of the 'Local' addition. Because if we used that, shouldn't we change the name up for MLOps.NET.Azure to MLOps.NET.RemoteStorage/Cloud.Azure. If we drop the '.Local", it will be more consistence with the current naming schema because the extension's name should be self-evident enough.

[aslotte]

Great input @sammysemantics. To address some of your points:

• I'm all good trying SQLite to start with
• Maybe the name of the package should be MLOps.NET.SQLite?

Like that we can have additional packages for e.g. MLOps.NET.AWS if we open up for AWS as a storage provider.

[aslotte]

It could also be called MLOps.NET.LocalStorage.SQLite and we could have MLOps.NET.LocalStorage.SqlServer etc.

[sammysemantics]

If we rename it MLOps.NET.LocalStorage.*, should we rename the Azure extension to MLOps.NET.RemoteStorage (or CloudStorage).Azure? Because I do see more extensions to this library that fits outside of storage and focuses on other operations in the MLOps lifecycle.

[aslotte]

@sammysemantics - what do you think about re-naming it to MLOps.NET.Storage.Azure and MLOps.NET.Storage.SQLite etc.?

[dcostea]

@sammysemantics - what do you think about re-naming it to MLOps.NET.Storage.Azure and MLOps.NET.Storage.SQLite etc.?

For me, it's better this way. Azure and SQLite are intrinsic local or remote, so there is no need to encode this in the namespace.

[aslotte]

@dcostea just so I understand your last comment, do you think they should be called MLOps.NET.Storage.Azure or MLOps.NET.Azure for example?

I personally prefer MLOps.NET.Azure but I'm good with either and want to make sure it makes sense to you, @sammysemantics and others.

[dcostea]

If we don't plan to have more modules related to Azure, then I prefer the shorter one, MLOps.NET.Azure. Similarly MLOps.NET.SQLite.

[sammysemantics]

Agreed.

[aslotte]

Perfect, that sounds great.