Support for generated columns

[simonw]

Refs #1116. My first attempt at this worked on my laptop but broke in CI, so I'm going to iterate on it in a pull request instead.

[simonw]

I need to replicate these failures on my laptop. My hunch is that this is down to the version of SQLite available to Python.

[simonw]

On my laptop:

platform darwin -- Python 3.8.6, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
SQLite: 3.33.0

In CI they are all SQLite: 3.22.0

[simonw]

This kind of problem is why I have a tmate workflow:

[simonw]

Here's the problem: https://www.sqlite.org/changes.html#version_3_26_0

2018-12-01 (3.26.0)

• Added PRAGMA table_xinfo that works just like PRAGMA table_info except that it also shows hidden columns in virtual tables.

CI is running 3.22.0.

[simonw]

I'm nervous about the performance impact of this, since every time it is called it creates a new in-memory connection.

I'm going to cache this so that it only ever gets called a maximum of once for the lifetime of the server.

[codecov]

Codecov Report

Merging #1117 (ccdf2c6) into main (dea3c50) will decrease coverage by 0.00%.
The diff coverage is 95.23%.

@@            Coverage Diff             @@
##             main    #1117      +/-   ##
==========================================
- Coverage   91.48%   91.48%   -0.01%     
==========================================
  Files          30       31       +1     
  Lines        3841     3852      +11     
==========================================
+ Hits         3514     3524      +10     
- Misses        327      328       +1     

Impacted Files	Coverage Δ
datasette/utils/__init__.py	94.10% <87.50%> (-0.20%)	⬇️
datasette/utils/sqlite.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dea3c50...ccdf2c6. Read the comment docs.

[nattaylor]

I just deployed this and its working great.

In a very unscientific benchmark my response times went from around 22-25ms to 33-36ms, but I didn't even dig enough to confirm the latency is related to the change. It's on a VPS, so maybe the load changed. I don't see any difference in performance.