Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for pydantic 2.0, polars 0.20.10 and remove duckdb support #32

Merged
merged 29 commits into from
Feb 27, 2024
Merged

Add support for pydantic 2.0, polars 0.20.10 and remove duckdb support #32

merged 29 commits into from
Feb 27, 2024

Commits on Oct 26, 2023

  1. Downgrade Sphinx to compatible version

    @JakobGM @brendancooley
    JakobGM authored and brendancooley committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    a5ddd94 View commit details
    Browse the repository at this point in the history

  2. very rough draft for pydantic v2

    @brendancooley
    brendancooley committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    2287188 View commit details
    Browse the repository at this point in the history

  3. fix: DataFrameValidationError mirrors pydantic v1 ValidationError 

    test_validators passes with exception of tests containing `json_schema_extra`
    @brendancooley
    brendancooley committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    42299db View commit details
    Browse the repository at this point in the history

  4. wip: subclass FieldInfo, extend with patito field attributes 

    all `test_validators` now succeed, all but `test_dataframe_get_method` now pass in `test_polars`
    @brendancooley
    brendancooley committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    4b98da9 View commit details
    Browse the repository at this point in the history

  5. fix: update LDF.collect() for polars==0.19.8 

    fix: switch to *args/**kwargs
    @brendancooley
    brendancooley committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    eef74fe View commit details
    Browse the repository at this point in the history

  6. wip: restructure model reconstitution routines, all tests passing

    @brendancooley
    brendancooley committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    87d57bd View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. chore: lockfile and formatting 

    chore: cleanup init

chore: cleanup init

chore: more misc cleanup

cleanup json_schema_extras
    @brendancooley
    brendancooley committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    ffe343d View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. fix: handle multiple annotations when dtype is present in Field

    @brendancooley
    brendancooley committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    e2bf0d7 View commit details
    Browse the repository at this point in the history

  2. check: annotated dtypes match those specified in Field.dtype

    @brendancooley
    brendancooley committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    017c59b View commit details
    Browse the repository at this point in the history

  3. chore: misc typing improvements

    @brendancooley
    brendancooley committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    161300b View commit details
    Browse the repository at this point in the history

  4. fix: search for nested field constraints on validation, ignore nulls

    @brendancooley
    brendancooley committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    9e132bc View commit details
    Browse the repository at this point in the history

  5. fix: better inference for column nullability

    @brendancooley
    brendancooley committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    93e171e View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. wip: better type mismatch messaging, move Representation to internal

    @brendancooley
    brendancooley committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    e9dbca6 View commit details
    Browse the repository at this point in the history

  2. wip: onto classproperty formulation in lieu of metaclass 

    not supported python<3.9, will break in python>=3.13
at that point, we can roll our own following polars: https://github.com/pola-rs/polars/blob/8d29d3cebec713363db4ad5d782c74047e24314d/py-polars/polars/datatypes/classes.py#L25C12-L25C12
    @brendancooley
    brendancooley committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    399673e View commit details
    Browse the repository at this point in the history

  3. wip: custom classproperty (from polars)

    @brendancooley
    brendancooley committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    a63db3f View commit details
    Browse the repository at this point in the history

  4. wip: robustify array dtype inference, add pt custom fields to Field()

    @brendancooley
    brendancooley committed Nov 6, 2023
    2 Configuration menu
    Copy the full SHA
    db44aa3 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. feature: custom Field/FieldInfo constructors for better extensibility

    @brendancooley
    brendancooley committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    438974c View commit details
    Browse the repository at this point in the history

  2. fix: make _PatitoFieldInfo private, ensure ext fields -> _schema_props

    @brendancooley
    brendancooley committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    386ba83 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. wip: serialization 

    wip: ColumnInfo API

cleanup
    @brendancooley
    brendancooley committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ee63672 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. wip: modularize annotation -> dtype and dtype validation 

    working to minimize references to _schema_properties and work with more structured and customizable objects. should be easier to add new supported dtypes with this framework and debug in future

wip: tests -> new dtype inference engine

wip: dtype docstrings
    @brendancooley
    brendancooley committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    daf8f59 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. chore: remove database/duckdb support

    @brendancooley
    brendancooley committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    054d034 View commit details
    Browse the repository at this point in the history

  2. chore: more sql cleanup

    @brendancooley
    brendancooley committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    155ce1f View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. chore: more migration/refactor 

    - more db cleanup
- column info -> _pydantic internals
- DTypeResolver
- cached schema helpers -> _pydantic internals
- (polars) derive, cast -> LDF
- validate_schema as classmethod
- time/timedelta -> example generators
    @brendancooley
    brendancooley committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    6221dd7 View commit details
    Browse the repository at this point in the history

  2. chore: add tests, some new features 

    - support validation for column subsets
- tests for nested models (as structs)
- fill_null adds missing columns with default values
- test recursive derivation
- test derive column subset
- allow conversion pt.DataFrame -> pl.DataFrame
- support pydantic validation_alias

chore: fixes for python 3.9 (all tests passing)

chore(patito): cleanup
    @brendancooley
    brendancooley committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    dbad4cf View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. bug: instantiated dtypes, constraints on examples, tz-less datetimes

    @brendancooley
    brendancooley committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    48ed782 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. chore: docs cleanup, doctests running, partial linting

    @brendancooley
    brendancooley committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    a2459a9 View commit details
    Browse the repository at this point in the history

  2. chore: wip types

    @brendancooley
    brendancooley committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    7836d16 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. bump version and poetry update

    @thomasaarholt
    thomasaarholt committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    7c36b72 View commit details
    Browse the repository at this point in the history

  2. Use pl.String instead of pl.Utf8

    @thomasaarholt
    thomasaarholt committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    b5c7002 View commit details
    Browse the repository at this point in the history