Release sqlite-utils 3.39

[simonw]

Mainly for this bug fix:

• Add pip as a dependency to ensure plugins work with uv #687

But I can bundle a few other small things in there too.

[simonw]

Looking at 3.38...4.0a1 a few stand out. These are features in the 4.0 alpha that do NOT break backwards compatibility:

• Whether or not a table remembers or detects its primary key is confusing and messy #655
• .insert_all() and .upsert_all() to support iterator of Python lists/tuples #672
• --functions should take filenames, and be usable multiple times #659

[simonw]

Also:

• 04107d3
• 88a83f0
• 72f6c82
• cf1b407

And maybe:

• Remove these Sentinel hacks when Click 8.3.1 ships #666

[simonw]

Told Claude Code:

create a new branch called 3.x starting with the 3.38 tag, then consult
https://github.com/simonw/sqlite-utils/issues/688 and cherry-pick the commits it lists in the second comment, then review each of the links in the first comment and cherry-pick those as well. After each cherry-pick run the command "just test" to confirm the tests pass and fix them if they don't. Look through the commit history on main since the 3.38 tag to help you with this task.

https://gistpreview.github.io/?83c7a7ea96d6b7763ad5d72d251ce1a6

It did most of the work and reported back:

I've successfully created the 3.x branch and cherry-picked most of the commits. Here's a summary of what was
completed:

Successfully Cherry-picked ✓

From the second comment (4 commits):

1. ✓ 04107d3 - Remove note from docs about 3.8 and deterministic (Drop Python 3.8 #646)
2. ✓ 88a83f0 - Link to JSON Lines instead (fixed expired ndjson.org domain)
3. ✓ 72f6c82 - Small documentation updates to fix warnings
4. ✓ cf1b407 - Add a type hint for db.close() (Add a type hint for db.close() #663)

From the first comment (issues):
5. ✓ a0d8bc3 - Add pip as a dependency to ensure plugin installation (#687) - the main bug fix
6. ✓ 0bbc680 - --functions can take filenames, can be used multiple times (#659)

All tests pass after each cherry-pick!

Remaining Issues (require manual review)

Two features from the first comment have complex merge conflicts with 3.38:

• Whether or not a table remembers or detects its primary key is confusing and messy #655 (table primary key detection): Commit 094b010 extensively refactors db.table() vs db.view() behavior
• .insert_all() and .upsert_all() to support iterator of Python lists/tuples #672 (generators for insert_all/upsert_all): Commit 7ffd505 has significant conflicts in core insertion logic

These would require substantial manual porting rather than clean cherry-picks. Given that the main purpose of 3.39 is
the #687 bug fix, I recommend:

• Release 3.39 with what we have (the bug fix plus the smaller backward-compatible improvements)
• Consider Whether or not a table remembers or detects its primary key is confusing and messy #655 and .insert_all() and .upsert_all() to support iterator of Python lists/tuples #672 for a future 3.40 release if needed

The 3.x branch is ready with 9 commits on top of 3.38!

Todos
☒ Fetch and review issue #688 to understand requirements
☒ Look at commit history on main since 3.38 tag
☒ Create 3.x branch starting from 3.38 tag
☒ Cherry-pick 04107d3 (docs: deterministic)
☒ Cherry-pick 88a83f0 (JSON Lines link)
☒ Cherry-pick 72f6c82 (docs warnings)
☒ Cherry-pick cf1b407 (db.close type hint)
☒ Cherry-pick a0d8bc3 (#687 pip dependency)
☒ Cherry-pick 0bbc680 (#659 --functions)
☐ Review remaining commits (#655, #672)

[simonw]

I'm convinced that it's not worth implementing #655 and #672, those can stay in the alpha.

[simonw]

I pushed those changes to https://github.com/simonw/sqlite-utils/compare/3.x

[simonw]

Turns out the Sentinel thing didn't need reverting as we never published that code to the 3.x branch int the first place. Pinning to click>=8.3.1 will fix that instead.

[simonw]

Reviewing this is a bit tricky. I'm resetting things so the 3.x branch is back to being what was in 3.38 and a new 3.39-fixes branch has the new changes. Then I can use a PR between 3.39-fixes and 3.x to review and fix up the new code.

[simonw]

OK I'm going to do the rest of the work in this new PR:

• Fixes for 3.39 #689

[simonw]

I'm going to drop Python 3.8 and 3.9 since they are EOL.

[simonw]

So the release notes need to talk about:

• --functions supporting file paths and multiple options
• Python 3.10 and up
• Fix for the sqlite-utils install bug

[simonw]

• Fixed a bug with sqlite-utils install when the tool had been installed using uv. Add pip as a dependency to ensure plugins work with uv #687
• The --functions argument now optionally accepts a path to a Python file as an alternative to a string full of code, and can be specified multiple times. --functions should take filenames, and be usable multiple times #659
• sqlite-utils now requires on Python 3.10 or higher.