Merge pull request #577 from oddbird/autocommit-update

SQLAlchemy updates

content/blog/2014/sqlalchemy-postgres-autocommit.md

@@ -20,6 +20,21 @@ summary: |
 date: 2014-06-14
 ---
 
+{% import 'utility.macros.njk' as utility %}
+
+{% set update = ['Update', utility.datetime('2024-04-26')] | join(' ') %}
+{% callout 'note', update %}
+This article was written before the release of SQLAlchemy 2.0. The library has
+changed significantly since then, and we have added relevant notes to the
+sections about [autocommit mode][am], [real autocommit], and [starting
+transactions]. TL;DR -- autocommit mode was deprecated and the isolation level
+approach is now recommended and thoroughly explained by the official docs.
+
+[am]: #sqlalchemy-s-autocommit-mode-not-what-you-think
+[real autocommit]: #turning-on-real-autocommit
+[starting transactions]: #starting-a-transaction
+{% endcallout %}
+
 [SQLAlchemy] defaults to implicitly opening a new transaction on your
 first database query. If you prefer to start your transactions
 explicitly instead, I've documented here my explorations in getting that
@@ -167,7 +182,9 @@ that's the behavior I want with SQLAlchemy. Can I make that work?
 
 [sqlalchemy]: https://www.sqlalchemy.org/
 
-### SQLAlchemy's Autocommit Mode -- Not What You Think
+<h3 id="sqlalchemy-s-autocommit-mode-not-what-you-think">
+  SQLAlchemy's Autocommit Mode -- Not What You Think
+</h3>
 
 I soon found [autocommit mode] in SQLAlchemy's documentation, and
 thought I had my answer -- but no such luck. SQLAlchemy's autocommit
@@ -180,7 +197,15 @@ unnecessary transactions.
 
 [autocommit mode]: https://docs.sqlalchemy.org/en/13/orm/session_transaction.html#autocommit-mode
 
-### Turning on Real Autocommit
+{% callout 'note', update %}
+Autocommit mode was deprecated in SQLAlchemy 1.4 and [removed in 2.0]. The
+approach explained in the next section is now the recommended way to achieve
+autocommit functionality, albeit with some changes.
+
+[removed in 2.0]: https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#library-level-but-not-driver-level-autocommit-removed-from-both-core-and-orm
+{% endcallout %}
+
+<h3 id="turning-on-real-autocommit">Turning on Real Autocommit</h3>
 
 Happily, setting all of SQLAlchemy's psycopg2 connections into real
 autocommit became quite easy in SQLAlchemy 0.8.2: SQLAlchemy's psycopg2
@@ -192,6 +217,14 @@ from sqlalchemy import create_engine
 engine = create_engine('postgresql://test', isolation_level="AUTOCOMMIT")
 ```
 
+{% callout 'note', update %}
+SQLAlchemy 2.0 supports this isolation level in both [engines] and individual
+[connections].
+
+[engines]: https://docs.sqlalchemy.org/en/20/core/connections.html#setting-isolation-level-or-dbapi-autocommit-for-an-engine
+[connections]: https://docs.sqlalchemy.org/en/20/core/connections.html#setting-isolation-level-or-dbapi-autocommit-for-a-connection
+{% endcallout %}
+
 We haven't discussed transaction isolation levels yet (and I won't in
 detail here). They control the visibility of changes between multiple
 concurrent transactions. The [Postgres documentation] summarizes the
@@ -211,7 +244,7 @@ mode.
 [postgres documentation]: https://www.postgresql.org/docs/9.2/transaction-iso.html
 [and psycopg2]: https://www.psycopg.org/docs/extensions.html#psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT
 
-### Starting a Transaction
+<h3 id="starting-a-transaction">Starting a Transaction</h3>
 
 If we didn't want to use transactions at all, this would be all we need.
 SQLAlchemy would happily hum along thinking it has a transaction but
@@ -226,6 +259,66 @@ the database. But we don't actually need to issue `BEGIN` ourselves
 either - we just need to turn off the `autocommit` property on our
 connection, and then `psycopg2` will issue the `BEGIN` for us.
 
+{% callout 'note', update %}
+SQLAlchemy 2.0 now [recommends] starting an "autocommit" connection when needed,
+and using the regular connections otherwise:
+
+[recommends]: https://docs.sqlalchemy.org/en/20/core/connections.html#changing-between-isolation-levels
+
+```python
+# autocommit block, no need to begin() or commit()
+with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as connection:
+    connection.execute(text("<statement>"))
+
+# regular block, needs begin() and calls commit() at exit
+with engine.begin() as connection:
+    connection.execute(text("<statement>"))
+```
+
+If you prefer to autocommit by default, you can spin off an "autocommit" engine
+and use it for all your sessions and connections, and call the regular engine
+when you need transactions:
+
+```python
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+
+transaction_engine = create_engine("postgresql//test")
+autocommit_engine = engine.execution_options(isolation_level="AUTOCOMMIT")
+
+TransactionSession = sessionmaker(bind=engine)
+AutocommitSession = sessionmaker(bind=autocommit_engine)
+
+# This session autocommits
+with AutocommitSession() as session:
+    session.add(some_object)
+    session.add(some_other_object)
+
+# This session needs begin()/commit()/rollback()
+with TransactionSession() as session:
+    session.begin()
+    try:
+        session.add(some_object)
+        session.add(some_other_object)
+    except:
+        session.rollback()
+        raise
+    else:
+        session.commit()
+
+# Or the equivalent shorthand version
+with TransactionSession.begin() as session:
+    session.add(some_object)
+    session.add(some_other_object)
+# Commits the transaction, closes the session
+```
+
+The rest of this article explains how to hook into the `begin()` call to turn an
+autocommit session into a transactional one in older versions of SQLAlchemy. As
+SQLAlchemy 2.0 explicitly [recommends] against this approach, we haven't updated
+it to work with newer versions.
+{% endcallout %}
+
 SQLAlchemy gives us a way to hook into the `begin()` call: the
 `after_begin` event, which sends along the relevant database connection.
 We have to dig through a few layers of connection-wrapping to get down

content/blog/2023/fastapi-path-operations-for-django-developers.md

@@ -15,6 +15,12 @@ summary: |
   perspective of a Django developer.
 ---
 
+{% callout 'note', false %}
+Check out our [Winging It](/wingingit/) channel for a conversation on FastAPI.
+
+**Winging It** episode 5: [Streamline Your API Development with FastAPI](/2024/03/21/winging-it-05/)
+{% endcallout %}
+
 If you've heard about [FastAPI], a modern and fast web framework for building
 APIs with Python, you might be wondering how it compares to Django, the most
 popular and mature web framework for Python. In this series, I will answer this
@@ -86,7 +92,7 @@ consider it a definitive solution.
 
 In the middle of all this, I kept hearing about FastAPI and how it was not only
 fast, but also leveraged Python's type system to provide a better developer
-experience *and* automatic documentation and schemas for API consumers. After
+experience _and_ automatic documentation and schemas for API consumers. After
 following its excellent [tutorial], I asked the team to consider it for
 [OddBooks], our collaborative writing tool. An exploratory branch was created
 and after reviewing the resulting code, we decided to go ahead and officially
@@ -190,10 +196,10 @@ def delete_version(version_id: int):
     delete_version_from_db(id=version_id)
 ```
 
-*Note: I'm hiding the actual database read and write operations behind
+_Note: I'm hiding the actual database read and write operations behind
 `get_versions_from_db` and similar functions. How you [connect to your database]
 is a separate topic and I want to focus on writing and consuming API endpoints
-here.*
+here._
 
 [connect to your database]: /2023/10/23/sqlalchemy-for-django-developers/
 
@@ -209,9 +215,9 @@ In contrast with the Django version, we get:
   has done away with a handful of unit / integration tests and consistently
   warns the frontend team when the API has changed.
 - Runtime validation of the request body and URL parameters by using type hints.
-  FastAPI will ensure that something like `def update_version(id: int, version:
-  VersionUpdate):` will only accept a JSON body with a `title` field and an
-  integer URL parameter.
+  FastAPI will ensure that something like
+  `def update_version(id: int, version: VersionUpdate):` will only accept a JSON
+  body with a `title` field and an integer URL parameter.
 - Automatic serialization of the response body by using the `response_model`
   parameter. FastAPI will ensure that the response body is a JSON object with
   the expected fields and types. The path operation itself can return anything

content/blog/2023/sqlalchemy-for-django-developers.md

@@ -18,6 +18,17 @@ summary: |
   queries with SQLAlchemy, and highlight key differences.
 ---
 
+{% import 'utility.macros.njk' as utility %}
+
+{% set update = ['Update', utility.datetime('2024-04-26')] | join(' ') %}
+{% callout 'note', update %}
+- Added a new section on [transactions].
+- Expanded information on [migrations].
+
+[transactions]: #transactions-are-on-by-default
+[migrations]: #migrations-are-not-built-in
+{% endcallout %}
+
 If you've heard about [FastAPI], a modern and fast web framework for building
 APIs with Python, you might be wondering how it compares to Django, the most
 popular and mature web framework for Python. In this series, I will answer this
@@ -78,9 +89,9 @@ class User(DeclarativeBase):
      fullname: Mapped[Optional[str]]
 ```
 
-*Note: this [declarative style] for model definition is relatively new,
+_Note: this [declarative style] for model definition is relatively new,
 superseding the old `declarative_base` function in SQLAlchemy 2.0. You might
-still encounter the old style in some codebases.*
+still encounter the old style in some codebases._
 
 [declarative style]: https://docs.sqlalchemy.org/en/20/orm/mapping_styles.html#orm-declarative-mapping
 
@@ -134,8 +145,8 @@ with Session(engine) as session:
     users = session.execute("SELECT * FROM users").all()
 ```
 
-*Notice we are using raw SQL here instead of the ORM. We will get to the ORM in
-the next section.*
+_Notice we are using raw SQL here instead of the ORM. We will get to the ORM in
+the next section._
 
 You don't need to use a context manager to create a session, but it is
 recommended so that the session is automatically closed when you are done with
@@ -192,14 +203,13 @@ achieves this by exposing custom methods as part of the class attributes:
 users = session.scalars(select(User).where(User.id.in_([1, 2, 3]))).all()
 ```
 
-*The trailing underscore in `in_()` is needed because `in` is a reserved word in
-Python, not because of anything specific to SQLAlchemy.*
+_The trailing underscore in `in_()` is needed because `in` is a reserved word in
+Python, not because of anything specific to SQLAlchemy._
 
 There's a whole host of interesting methods you can use with model attributes as
 explained in the [`ColumnElement` documentation].
 
-[`ColumnElement` documentation]:
-    https://docs.sqlalchemy.org/en/20/core/sqlelement.html#sqlalchemy.sql.expression.ColumnElement
+[`ColumnElement` documentation]: https://docs.sqlalchemy.org/en/20/core/sqlelement.html#sqlalchemy.sql.expression.ColumnElement
 
 The `select` function accepts entire model classes or individual columns as
 arguments. For example, to get only the `name` column, you can do
@@ -233,9 +243,43 @@ session.add_all([user1, user2, address1])
 session.commit()
 ```
 
-In Django terms, the session is like a transaction that you can commit to when
-you're ready, and the notion of saving individual model instances by calling one
-of their methods is not present.
+The notion of saving individual model instances by calling one of their methods
+is not present because transactions are always enabled, as we'll see in the next
+section.
+
+## Transactions are On by Default
+
+In Django, you need to wrap your database operations in a transaction to ensure
+that they are atomic. This means that if an exception is raised during the
+operation, the changes are rolled back. This is done by using the `atomic`
+decorator or the `transaction.atomic` context manager:
+
+```python
+from django.db import transaction
+
+with transaction.atomic():
+    User.objects.create(name="John")
+```
+
+In SQLAlchemy, transactions are on by default. This means that every operation
+you perform is wrapped in a transaction. You can commit the transaction
+explicitly with `session.commit()` or rollback with `session.rollback()`. If you
+don't commit the transaction, the changes will be rolled back when the session
+is closed.
+
+```python
+from sqlalchemy.orm import Session
+
+with Session(engine) as session:
+    session.execute("INSERT INTO users (name) VALUES ('John')")
+    session.commit()
+```
+
+If you prefer the Django way of doing things, it's actually possible to create
+an "autocommit" engine in SQLAlchemy. Refer to our article on [SQLAlchemy
+Autocommit] for more information.
+
+[SQLAlchemy Autocommit]: /2014/06/14/sqlalchemy-postgres-autocommit
 
 ## Relations Require More Work
 
@@ -339,9 +383,9 @@ We won't go into details here, but the basic substitutions are:
 
 - `./manage.py makemigrations` becomes `alembic revision --autogenerate`
 - `./manage.py migrate` becomes `alembic upgrade head`
-- `./manage.py migrate app <migration number>` becomes `alembic upgrade
-  <revision hash>` if going forward, or `alembic downgrade <revision hash>` if
-  going back
+- `./manage.py migrate app <migration number>` becomes
+  `alembic upgrade <revision hash>` if going forward, or
+  `alembic downgrade <revision hash>` if going back
 
 SQLAlchemy and Alembic don't have the concept of "apps" as standalone elements
 with their own models and migrations. Instead, they use a single
@@ -353,6 +397,13 @@ defined by third-party packages. This is in contrast with Django where
 third-parties usually ship their own migration history to manage their tables
 independently from user-defined models.
 
+Alembic is also less helpful when it comes to automatically generating
+revisions. Some column attributes, like `server_default`, seem to be completely
+ignored when it comes to detecting changes in the model. When adding new columns
+to existing tables, Alembic will not warn you if the new column is not nullable.
+This can lead to data integrity issues if you are not careful, and the decision
+of adding a default or somehow populating your old rows is left to you.
+
 ## Conclusion
 
 SQLAlchemy is a powerful library. We have only scratched the surface of what it

src/filters/type.js

@@ -222,5 +222,5 @@ export const callout = (content, type = 'note', label = true) => {
 
   return `<div data-callout="${type}">${
     displayLabel ? `<strong>${displayLabel}:</strong>` : ''
-  }<div>${md(content.trim())}</div></div>`;
+  }<div>\n\n${content}\n\n</div></div>`;
 };

src/scss/patterns/_callout.scss

@@ -1,5 +1,6 @@
 [data-callout] {
   --list-padding--default: 1em;
+  --inline-bleed: 0;
 
   background: var(--callout-block-bg, var(--callout));
   border-inline-start: thick solid var(--callout-block-border, var(--border));