Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-42733: minor improvement spin-offs from query system rewrite #954

Merged
merged 17 commits into from
Feb 8, 2024
Merged

DM-42733: minor improvement spin-offs from query system rewrite #954

merged 17 commits into from
Feb 8, 2024

Commits on Feb 8, 2024

  1. Fix "/" -> "*" typo in query expression evaluation. 

    Apparently nobody has ever tried to multiply anything inside a butler
query expression, because it wouldn't have worked.

Note that we don't actually need an entry for "/" in this mapping
because that requires special-casing to deal with floating-point vs.
integer division anyway.
    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    4b9aa31 View commit details
    Browse the repository at this point in the history

  2. Add wrapper class that defers pydantic validation. 

    The wrapper class itself is gnarly and full of advanced Pydantic usage
and metaprogramming, but it's general and it keeps the messiness
outside of both the serialized types and the validated types without
duplication or two-stage validation.
    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    c619da4 View commit details
    Browse the repository at this point in the history

  3. Add function for getting DimensionUniverse from validation context.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    ac84152 View commit details
    Browse the repository at this point in the history

  4. Add pydantic serialization for DimensionGroup.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    95f2436 View commit details
    Browse the repository at this point in the history

  5. Add Pydantic serialization for DimensionElement.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    bd95eab View commit details
    Browse the repository at this point in the history

  6. Add ColumnSpec and ToArrow variants for astropy.time.Time.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    3c319db View commit details
    Browse the repository at this point in the history

  7. Enumerate supported column types in a single Literal type alias.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    18fdce4 View commit details
    Browse the repository at this point in the history

  8. Add ColumnSpec and ToArrow variants for uuid.UUID.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    ac01be3 View commit details
    Browse the repository at this point in the history

  9. Allow limit=None (sort of) but not offset=None. 

    We mixed up which of these should be allowed to be None in the old
interface, but we don't need to repeat the mistake in the new one,
though some of the fix will have to wait until new interfaces stop
delegating to the old ones.
    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    4b73e3c View commit details
    Browse the repository at this point in the history

  10. Add NonemptyMapping utility class.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    2ccf797 View commit details
    Browse the repository at this point in the history

  11. Get PostgreSQL server version on engine startup.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    a308fbc View commit details
    Browse the repository at this point in the history

  12. Add SQLAlchemy aggregate function for region unions. 

    We just concatenate the encoded strings with a ":" (could be anything
that isn't in the base64 alphabet) then split them in Python before
decoding.

This will let use GROUP BY on dimension key columns for deduplication
while preserving region columns needed for Python-side region-overlap
postprocessing.

Rename union_agg to union_aggregate.
    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    6b5f2fd View commit details
    Browse the repository at this point in the history

  13. Add feature flags and abstraction for DISTINCT ON and ANY_VALUE. 

    SQLite does not require SELECT columns to be in aggregates if they are
not in the GROUP BY clause.  What PostgreSQL can do depends on the
version.
    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    9fc0fa1 View commit details
    Browse the repository at this point in the history

  14. Add TimespanDatabaseRepresentation.apply_any_aggregate.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    cd1727d View commit details
    Browse the repository at this point in the history

  15. Add tests for new SQL aggregate functionality.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    a31864d View commit details
    Browse the repository at this point in the history

  16. Drop removed symbol from doc build skips.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    38c3e67 View commit details
    Browse the repository at this point in the history

  17. Add some previously-omitted modules to doc build.

    @TallJimbo
    TallJimbo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    fb90c4a View commit details
    Browse the repository at this point in the history