Support SQLite for custom data layer

[tjroamer]

Overview

• Adds custom SQLite data layer, which is very simple to set up, no need to create extra database!
• This datalayer inherits from ChainlitDataLayer, reusing its business logic and re-implements API functions of LiteralClient
• Inspired by @sandangel work on Support MongoDB for custom data layer #796 for MongoDB
• Inspired by @hayescode work on Support Postgres for custom data layer #825 for Postgres

How to configure

import chainlit.data as cl_data
from chainlit.data.sqlite import SQLiteDataLayer

# a default database file chainlit.db will be created in the working dir
cl_data._data_layer = SQLiteDataLayer()

# Or optionally provide own file (can be relative or absolute) as the init parameter
# this code will create C:/myfolder if it doesn't exist, and data is persited in mydata.db
cl_data._data_layer = SQLiteDataLayer(db_filepath="C:/myfolder/mydata.db")

Test

A simple sanity check is implemented in cypress/e2e/data_layer_sqlite, which is a simplified copy of cypress/e2e/data_layer.

Limitation

no file blob support, so chat history will not show attachments

[Rajatkhanna801]

Hi @tjroamer
I want to know If I want to create separate sqlite database for each user how I can do that.

[tjroamer]

Hi @tjroamer I want to know If I want to create separate sqlite database for each user how I can do that.

With the current chainlit data layer, user-specific sqlite database might not be straightforward to implement. One solution I can think of is that you create views for different users, and select only rows that relevant to that user. This would require some post-processing steps once you have the database as a single file.

[Rajatkhanna801]

@tjroamer
Actually as per my requirement I want separate sqlite database for each user and I think we can achieve that with sending custom database url.

[tjroamer]

Will these users use the same chatbot based on the same server? If this is the case, you cannot send custom database url for each user, since the database url is a server-side setting, and it is done only once when the server starts.

[Rajatkhanna801]

okay let me figure another way for that.

[Rajatkhanna801]

@tjroamer
I have configured Microsoft authentication and everytime new user logged in I am creating new database sqlite file.

But I am not sure how I can put data in that tables for specific user.

[intc-hharshtk]

I just see this when I load old thread How to fix it