feature: Distribute Meltano with a dependency on a compatible SQLite binary

[edgarrmondragon]

Meltano should be shipped with the version of SQLite it expects to work with fine out of the box. Adding a dependency on https://pypi.org/project/pysqlite3-binary might be worth exploring, although it's lacking wheels for macOS and Windows at the moment.

Originally posted by @edgarrmondragon in #6405 (comment)

[stale]

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

[WillDaSilva]

This is still relevant. Today while running Meltano in an AWS Lambda Docker container I received this error message:

Detected SQLite 3.7.17, but Meltano requires at least 3.25.1. Upgrade your database to be compatible with Meltano or use a different database.

It'd certainly be nice if Meltano didn't have this implicit dependency.

[edgarrmondragon]

Too bad the maintainer doesn't want to publish macOS and Windows wheels at the moment: coleifer/pysqlite3#42 (comment)

[edgarrmondragon]

sqlean.py seems like a good alternative

[edgarrmondragon]

Ok, here's an approach that works:

pipx install meltano --python python3.10 --suffix=@sqlean
pipx inject meltano@sqlean sqlean-driver
meltano@sqlean init poc-sqlean
cd poc-sqlean
meltano@sqlean config config meltano set database_uri 'sqlite+sqlean:///$MELTANO_SYS_DIR_ROOT/meltano.db' --store=meltano_yml

to confirm that we got a recent SQLite:

$ ~/.local/pipx/venvs/meltano@sqlean/bin/python -c 'import sqlean; print(sqlean.sqlite_version)'
3.42.0