Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-41158: (mostly) implement QueryDriver for DirectButler #915

Merged
merged 23 commits into from
Apr 5, 2024
Merged

DM-41158: (mostly) implement QueryDriver for DirectButler #915

merged 23 commits into from
Apr 5, 2024

Commits on Apr 4, 2024

  1. Move NameShrinker out of registry and add container methods.

    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    61f438d View commit details
    Browse the repository at this point in the history

  2. Accept NameShrinker in ColumnSpec.to_sql_spec.

    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    6ebc934 View commit details
    Browse the repository at this point in the history

  3. Guard against long field names in dimensions manager. 

    It is very unlikely that we'll ever trip this guard.
    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    2504bcf View commit details
    Browse the repository at this point in the history

  4. Default QueryTree argument to Query constructor. 

    This reduces the need to import more stuff from the queries subpackage
in butlers.
    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    ece2431 View commit details
    Browse the repository at this point in the history

  5. Move 'dimensions' attribute out of QueryBase/Query. 

    Query already has 'constraint_dimensions' (the full dimensions that
can be used for e.g. WHERE expressions), which is different from
what 'dimensions' should have meant (the dimensions of the output
objects) on the QueryResults classes.
    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    ec57484 View commit details
    Browse the repository at this point in the history

  6. Fix truncated docstring in queries.tree.ColumnSet.

    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    3c04617 View commit details
    Browse the repository at this point in the history

  7. Make queries.ExpressionFactory friendlier to type checkers. 

    Previous annotations rejected a lot of typical usage that would almost
always be fine (because most DimensionElements are Dimensions and are
hence usable directly as values).
    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    53092e0 View commit details
    Browse the repository at this point in the history

  8. Fix doc typo that inverted meaning in Database interface.

    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    08639c7 View commit details
    Browse the repository at this point in the history

  9. Guard against skypix regions in result specs. 

    SkyPix regions should always be computed on the fly in the client.
    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    c73556e View commit details
    Browse the repository at this point in the history

  10. Export InvalidQueryError at package scope.

    @TallJimbo
    TallJimbo committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    0015626 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Implement QueryDriver for DirectButler.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    2546a02 View commit details
    Browse the repository at this point in the history

  2. Implement Butler._query.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    b76888c View commit details
    Browse the repository at this point in the history

  3. Fix typo in comment.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    722ae4c View commit details
    Browse the repository at this point in the history

  4. Don't lift 'queries' symbols into package scope (yet). 

    They're not public, and some of them clash with Registry symbols.
    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    1c3729e View commit details
    Browse the repository at this point in the history

  5. Fix incorrect query-result type imports. 

    This test wanted the result objects from

   lsst.daf.butler.registry.queries

for use in annotations, but it was getting them from

   lsst.daf.butler.queries

and we didn't notice because we don't run MyPy on tests.
    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    b29f78a View commit details
    Browse the repository at this point in the history

  6. Reduce duplication in query result processing. 

    Co-authored-by: David H. Irving <david.irving@noirlab.edu>
    @TallJimbo @dhirving
    TallJimbo and dhirving committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    2ca5451 View commit details
    Browse the repository at this point in the history

  7. Open caching context in query context.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    3852316 View commit details
    Browse the repository at this point in the history

  8. Fix incorrect code comment about query transactions.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    b8d4454 View commit details
    Browse the repository at this point in the history

  9. Use dataclass for DirectQueryDriver materialization state.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    f045ba3 View commit details
    Browse the repository at this point in the history

  10. Drop redundant cursor close.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    98fe0c7 View commit details
    Browse the repository at this point in the history

  11. Defer guards on QueryBuilder distinct vs. group_by to make them useful.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    e4b6b13 View commit details
    Browse the repository at this point in the history

  12. Assert on rather than cast SqlColumnVisitor types.

    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    f5a5b56 View commit details
    Browse the repository at this point in the history

  13. Add test for DirectQueryDriver with PostgreSQL. 

    This has some failures that I've marked as expected until we try to fix
them again on DM-43697.
    @TallJimbo
    TallJimbo committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    8a03dd5 View commit details
    Browse the repository at this point in the history