Add an interactive shell for Sqlite3

[rhettinger]

BPO	33436
Nosy	@rhettinger, @bitdancer, @serhiy-storchaka, @erlend-aasland, @DiddiLeija
Files	sqlite3_repl.py: Minimal proof-of-concept 0001-Add-proof-of-concept-REPL.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-05-07.01:47:00.123>
labels = ['3.8', '3.9', '3.10', '3.11', 'type-feature', 'library']
title = 'Add an interactive shell for Sqlite3'
updated_at = <Date 2022-02-11.14:45:10.289>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2022-02-11.14:45:10.289>
actor = 'DiddiLeija'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-05-07.01:47:00.123>
creator = 'rhettinger'
dependencies = []
files = ['47573', '49199']
hgrepos = []
issue_num = 33436
keywords = ['patch']
message_count = 6.0
messages = ['316248', '316257', '316261', '327252', '370180', '413072']
nosy_count = 6.0
nosy_names = ['rhettinger', 'r.david.murray', 'serhiy.storchaka', 'Big Stone', 'erlendaasland', 'DiddiLeija']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue33436'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[rhettinger]

To facilitate rapid experimentation with SQL queries, add a shell to run commands and display results.

Attached is minimal proof-of-concept modeled loosely on the Linux sqlite3 REPL. Here's a sample session:

SQLite version 2.6.0

 Enter "help" for usage hints

sqlite> open Irises.db
sqlite> select distinct species from irises
('Iris-setosa',)
('Iris-versicolor',)
('Iris-virginica',)
sqlite> select species, avg(pet_len), avg(sep_len) from irises group by species
('Iris-setosa', 1.464, 5.005999999999999)
('Iris-versicolor', 4.26, 5.936)
('Iris-virginica', 5.552, 6.587999999999998)
sqlite> select count(*) from irises
(150,)
sqlite> quit

[bitdancer]

It's not immediately obvious what benefits this provides over the existing sqlite-native repl and the python repl. Can you expand a bit on the use case? Also, I seem to remember that Catherine Devlin had at one point thought about expanding her sqlpython to be multi-db capable, and while it doesn't look like that has happened, maybe that would be an alternative worth investigating? (I've never used it or looked at its code, so it may not be.)

[serhiy-storchaka]

It could be added in Tools/scripts directory or as a part of the sqlite3 module.

It's not immediately obvious what benefits this provides over the existing sqlite-native repl and the python repl.

The existing sqlite-native repl is not always installed when Python is installed (especially on Windows). It is handy that Python provides CLI to some modules, including zipfile and tarfile.

I would prefer CLI, but maybe both interfaces can be combined.

It would be nice to provide also a graphical Sqlite3 browser, which can be ran as a separate application and be integrated with IDLE (as turtledemo).

[BigStone]

as a graphical sqlite browser, you have sqlite_bro that may give you inspiration. It's about one single file.

[erlend-aasland]

I think this is a good idea. Proof-of-concept implementation added (invoke REPL with python -m sqlite3).

Possible improvements:
- Use sqlite3.complete_statement()
- Support Python syntax as well. For example:
>>> select * from t
>>> res = _
>>> print(res[0])

[DiddiLeija]

Do we still want to do this? See https://discuss.python.org/t/titling-sqlite3-table-columns-with-number/13656/3.

[erlend-aasland]

@rhettinger, @serhiy-storchaka, are you still +1 on this? I'm +1, because...

• implementation is easy; low maintenance cost
• I'm a fan of built-in tools like this (python -m calendar is one of my favourites)
• there has been some interest in this feature (re. the linked Discourse topic)

[serhiy-storchaka]

I am +1 on this for the same reasons. Go ahead, it would be fun.

[kumaraditya303]

I am +1, this would make Windows sqlite users happy.

[erlend-aasland]

I'll revive my patch.

[erlend-aasland]

I'll revive my patch.

Rebased and cleaned up slightly. Draft PR up: #95026

IMO, let's just provide a dead simple sqlite3 shell for now; if more features are needed, let's add those in separate issues.

[erlend-aasland]

Possible future enhancements:

• provide switches for overriding row and text factories
• add more SQLite CLI features:
  • -cmd COMMAND: run "COMMAND" before reading stdin
  • -deserialize: open the database using sqlite3_deserialize()
  • -bail: stop after hitting an error
  • -readonly: open the database read-only
  • .trace command: set trace callback to print
  • .progress command: adds a basic progress handler
  • .connection [close] [#]: Open or close an auxiliary database connection

[erlend-aasland]

Thanks Raymond, for the pitch. Thanks Dong-hee, Kumar, and Serhiy for helpful reviews. This has now been implemented by #95026. Enhancement requests go as separate issues.