feat: Add support for declarative views.

README.md

@@ -126,43 +126,56 @@ include database specific objects.
 
 ## Alembic-utils
 
-Currently, the set of supported declarative objects is essentially non-overlapping with
+Currently, the set of supported declarative objects is largely non-overlapping with
 [Alembic-utils](https://github.com/olirice/alembic_utils). However in principle, there's
-no reason that objects supported by this library couldn't begin to overlap (views, functions,
+no reason that objects supported by this library couldn't begin to overlap (functions,
 triggers); and one might begin to question when to use which library.
 
-First, it's likely that this library can/should grow handlers for objects already supported by
-alembic-utils. In particular, it's likely that any future support in this library for something
-like a view could easily accept an `alembic_utils.pg_view.PGView` definition and handle it directly.
-The two libraries are likely fairly complementary in that way, although it's important to note
-some of the differences.
+Note that where possible this library tries to support alembic-utils native objects
+as stand-ins for the objects defined in this library. For example, `alembic_utils.pg_view.PGView`
+can be declared instead of a `sqlalchemy_declarative_extensions.View`, and we will internally
+coerce it into the appropriate type. Hopefully this eases any transitional costs, or
+issues using one or the other library.
 
 Alembic utils:
 
-- Is more directly tied to Alembic and specifically provides functionality for autogenerating
-  DDL for alembic, as the name might imply. It does **not** register into sqlalchemy's event
-  system.
-- Requires one to explicitly find/include the objects one wants to track with alembic.
-- It provides direct translation of individual entities (like a single, specific `PGGrantTable`).
-- In most cases, it appears to define a very "literal" interface (for example, `PGView` accepts
-  the whole view definition as a raw literal string), rather than an abstracted one.
+1. Is more directly tied to Alembic and specifically provides functionality for autogenerating
+   DDL for alembic, as the name might imply. It does **not** register into sqlalchemy's event
+   system.
+
+2. Requires one to explicitly find/include the objects one wants to track with alembic.
+
+3. Declares single, specific object instances (like a single, specific `PGGrantTable`). This
+   has the side-effect that it can only track included objects. It cannot, for example,
+   remove objects which should not exist due to their omission.
+
+4. In most cases, it appears to define a very "literal" interface (for example, `PGView` accepts
+   the whole view definition as a raw literal string), rather than attempting to either abstract
+   the objects or accept abstracted (like a `select` object) definition.
+
+5. Appears to only be interested in supporting PostgreSQL.
 
 By contrast, this library:
 
-- SqlAlchemy is the main dependency and registration point. The primary function of the library
-  is to register into sqlalchemy's event system to ensure that a `metadata.create_all` performs
-  the requisite statements to ensure the state of the database matches the declaration.
-
-  This library does **not** require alembic, but it does (optionally) perform a similar function
-  by way of enabling autogeneration support for non-native objects.
-
-- Perhaps a technical detail, but this library registers the declaratively stated objects directly
-  on the metadata/declarative-base. This allows the library to automatically know the intended
-  state of the world, rather than needing to discover objects.
-- The intended purpose of the supported objects is to declare what the state of the world **should**
-  look like. Therefore the function of this library includes the (optional) **removal** of objects
-  detected to exist which are not declared (much like alembic does for tables). Whereas alembic-utils
-  only operates on objects you create entities for.
-- As much as possible, this library provides more abstracted interfaces for defining objects.
-  This is particularly important for objects like roles/grants where not every operation is a create
-  or delete (in contrast to something like a view).
+1. SqlAlchemy is the main dependency and registration point (Alembic is, in fact, an optional dependency).
+   The primary function of the library is to declare the underlying objects. And then registration into
+   sqlalchemy's event system, or registration into alembic's detection system are both optional features.
+
+2. Perhaps a technical detail, but this library registers the declaratively stated objects directly
+   on the metadata/declarative-base. This allows the library to automatically know the intended
+   state of the world, rather than needing to discover objects.
+
+3. The intended purpose of the supported objects is to declare what the state of the world **should**
+   look like. Therefore the function of this library includes the (optional) **removal** of objects
+   detected to exist which are not declared (much like alembic does for tables).
+
+4. As much as possible, this library provides more abstracted interfaces for defining objects.
+   This is particularly important for objects like roles/grants where not every operation is a create
+   or delete (in contrast to something like a view), where a raw SQL string makes it impossible to
+   diff two different a-like objects.
+
+5. Tries to define functionality in cross-dialect terms and only where required farm details out to
+   dialect-specific handlers. Not to claim that all dialects are treated equally (currently only
+   PostgreSQL has first-class support), but technically, there should be no reason we wouldn't support
+   any supportable dialect. Today SQLite (for whatever that's worth), and MySQL have **some** level
+   of support.

docs/source/api.md

@@ -1,14 +1,14 @@
 # API
 
-## Schemas
-
 ```{eval-rst}
 .. autoapimodule:: sqlalchemy_declarative_extensions.schema
-   :members:
-   :noindex:
+   :members: Schema, Schemas
 ```
 
-## Roles
+```{eval-rst}
+.. autoapimodule:: sqlalchemy_declarative_extensions.view
+   :members: Views, View, view, register_view
+```
 
 ```{eval-rst}
 .. autoapimodule:: sqlalchemy_declarative_extensions.role.base
@@ -20,16 +20,19 @@
    :members: Role
 ```
 
-## Grants
-
 ```{eval-rst}
 .. autoapimodule:: sqlalchemy_declarative_extensions.grant
    :members: Grants
 ```
 
-## Rows
-
 ```{eval-rst}
 .. autoapimodule:: sqlalchemy_declarative_extensions.row
+   :members: Row, Rows
+```
+
+## Alembic
+
+```{eval-rst}
+.. autoapimodule:: sqlalchemy_declarative_extensions.alembic
    :members:
 ```

docs/source/index.md

@@ -16,6 +16,7 @@ API <api>
 :hidden:
 
 Schemas <schemas>
+Views <views>
 Roles <roles>
 Grants <grants>
 Rows <rows>

docs/source/views.md

@@ -0,0 +1,161 @@
+# Views
+
+Views definition and registration can be performed exactly as it is done with other object
+types, by defining the set of views on the `MetaData` or declarative base, like so:
+
+```python
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy_declarative_extensions import declarative_database, View, Views
+
+_Base = declarative_base()
+
+
+@declarative_database()
+class Base(_Base):
+    __abstract__ = True
+
+    views = Views().are(
+        View("foo", "select * from bar where id > 10", schema="baz"),
+    )
+```
+
+And if you want to define views using raw strings, or otherwise not reference the tables
+produced off the `MetaData`, then this is absolutely a valid way to organize.
+
+## The `view` decorator
+
+However views differ from most of the other object types, in that they are convenient to
+define **in terms of** the tables they reference (i.e. your existing set of models/tables).
+In fact personally, all of my views are produced from [select](sqlalchemy.sql.expression.select) expressions
+referencing the underlying [Table](sqlalchemy.schema.Table) object.
+
+This commonly introduce a circular reference problem wherein your tables/models are defined
+through subclassing the declarative base, which means your declarative base cannot then
+have the views statically defined **on** the base (while simultaneously referencing those models).
+
+```{note}
+There are ways of working around this in SQLAlchemy-land. For example by creating a ``MetaData``
+ahead of time and defining all models in terms of their underlying ``Table``.
+
+Or perhaps by using SQLAlchemy's mapper apis such that you're not subclassing the declarative base
+for models.
+
+In any case, these options are more complex and probably atypical. As such, we cannot assume
+you will adopt them.
+```
+
+For everyone else, the [view](sqlalchemy_declarative_extensions.view) decorator is meant to be the
+solution to that problem.
+
+This strategy allows one to organize their views alongside the models/tables those
+views happen to be referencing, without requiring the view be importable at MetaData/model base
+definition time.
+
+### Option 1
+
+```python
+from sqlalchemy import Column, types, select
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy_declarative_extensions import view
+
+Base = declarative_base()
+
+
+class Foo(Base):
+    __tablename__ = 'foo'
+
+    id = Column(types.Integer, primary_key=True)
+
+
+@view()
+class Bar1(Base):
+    __tablename__ = 'bar'
+    __view__ = select(Foo.__table__).where(Foo.__table__.id > 10)
+
+    id = Column(types.Integer, primary_key=True)
+```
+
+The primary difference between Options 1 and 2 in the above example is of how the
+resulting classes are seen by SQLAlchemy/Alembic natively.
+
+In the case of `Bar1`, SQLAlchemy/Alembic actually think that class is a normal table.
+Therefore querying the view looks identical to a real table: `session.query(Bar1).all()`
+
+For alembic, this means that alembic thinks you defined a table and will attempt to
+autogenerate it (while this library will also notice it and attempt to autogenerate
+a conflicting view.
+
+In order to use this option, we suggest you use one or both of some utility functions provided
+under the `sqlalchemy_declarative_extensions.alembic`: [ignore_view_tables](sqlalchemy_declarative_extensions.alembic.ignore_view_tables)
+and [compose_include_object_callbacks](sqlalchemy_declarative_extensions.alembic.compose_include_object_callbacks).
+
+Somewhere in your Alembic `env.py`, you will have a block which looks like this:
+
+```python
+with connectable.connect() as connection:
+    context.configure(
+        connection=connection,
+        target_metadata=target_metadata,
+        ...
+    )
+```
+
+The above call to `configure` accepts an `include_object`, which tells alembic to include or ignore
+all detected objects.
+
+```python
+from sqlalchemy_declarative_extensions.alembic import ignore_view_tables
+...
+context.configure(..., include_object=ignore_view_tables)
+```
+
+If you happen to already be using `include_object` to perform filtering, we provide an additional
+utility to more easily compose our version with your own. Although you can certainly manually call
+`ignore_view_tables` directly, yourself.
+
+```python
+from sqlalchemy_declarative_extensions.alembic import ignore_view_tables, compose_include_object_callbacks
+...
+def my_include_object(object, *_):
+    if object.name != 'foo':
+        return True
+    return False
+
+context.configure(..., include_object=compose_include_object_callbacks(my_include_object, ignore_view_tables))
+```
+
+## Option 2
+
+```python
+from sqlalchemy import Column, types, select
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy_declarative_extensions import view
+
+Base = declarative_base()
+
+
+class Foo(Base):
+    __tablename__ = 'foo'
+
+    id = Column(types.Integer, primary_key=True)
+
+
+@view(Base)  # or `@view(Base.metadata)`
+class Bar2:
+    __tablename__ = 'bar'
+    __view__ = select(Foo.__table__).where(Foo.__table__.id > 10)
+```
+
+By contrast, with Option 2, your class is not subclassing `Base`, therefore it's
+not registered as a real table by SQLAlchemy or Alembic. There's no additional
+work required to get them to ignore the table, because it's not one.
+
+Unfortunately, that means you cannot **invisibly** treat it as though it's a normal model,
+largely because it doesn't have the columns enumerated out in the same way.
+
+However we can provide some basic support for treating it as a table as far as the ORM is concerned.
+For example, you can still `session.query(Bar2).all()` directly.
+
+However, in most cases views primarily benefit non-code consumers of the database, because there's
+no practical difference between querying a literal view, versus executing the underlying query
+of that view, through something like `session.execute(Bar2.__view__)`.

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "sqlalchemy-declarative-extensions"
-version = "0.2.3"
+version = "0.3.0"
 description = "Library to declare additional kinds of objects not natively supported by SqlAlchemy/Alembic."
 
 authors = ["Dan Cardin <ddcardin@gmail.com>"]
@@ -103,6 +103,7 @@ log_cli_level = 'WARNING'
 
 [tool.ruff]
 src = ["src", "tests"]
+
 target-version = "py37"
 select = ["C", "D", "E", "F", "I", "N", "Q", "RET", "RUF", "S", "T", "U", "YTT"]
 ignore = ["C901", "E501", "S101"]
@@ -123,6 +124,9 @@ extend-ignore = [
     "D413",
 ]
 
+[tool.ruff.isort]
+known-first-party = ["sqlalchemy_declarative_extensions", "tests"]
+
 [tool.ruff.per-file-ignores]
 "tests/**/*.py" = ["D", "S"]
 

src/sqlalchemy_declarative_extensions/__init__.py

@@ -1,17 +1,16 @@
 from sqlalchemy_declarative_extensions import dialects
 from sqlalchemy_declarative_extensions.alembic import register_alembic_events
-from sqlalchemy_declarative_extensions.grant import Grants
-from sqlalchemy_declarative_extensions.role import Role
-from sqlalchemy_declarative_extensions.role.base import Roles
-from sqlalchemy_declarative_extensions.row import Row, Rows
-from sqlalchemy_declarative_extensions.schema import Schema, Schemas
-
-# isort: split
 from sqlalchemy_declarative_extensions.api import (
     declarative_database,
     declare_database,
     register_sqlalchemy_events,
 )
+from sqlalchemy_declarative_extensions.grant import Grants
+from sqlalchemy_declarative_extensions.role import Role
+from sqlalchemy_declarative_extensions.role.base import Roles
+from sqlalchemy_declarative_extensions.row import Row, Rows
+from sqlalchemy_declarative_extensions.schema import Schema, Schemas
+from sqlalchemy_declarative_extensions.view.base import View, Views, view
 
 __all__ = [
     "declarative_database",
@@ -27,4 +26,7 @@
     "Roles",
     "Schema",
     "Schemas",
+    "view",
+    "View",
+    "Views",
 ]

src/sqlalchemy_declarative_extensions/alembic/__init__.py

@@ -1,5 +1,11 @@
-from sqlalchemy_declarative_extensions.alembic.base import register_alembic_events
+from sqlalchemy_declarative_extensions.alembic.base import (
+    compose_include_object_callbacks,
+    ignore_view_tables,
+    register_alembic_events,
+)
 
 __all__ = [
     "register_alembic_events",
+    "ignore_view_tables",
+    "compose_include_object_callbacks",
 ]