Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] DataAsset uses partitioner from BatchConfig #9499

Merged
merged 75 commits into from
Feb 26, 2024
Merged

[FEATURE] DataAsset uses partitioner from BatchConfig #9499

merged 75 commits into from
Feb 26, 2024

Commits on Feb 14, 2024

  1. wip

    @joshua-stauffer
    joshua-stauffer committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    c3969dd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into f/v1-175/generic_partitioners

    @joshua-stauffer
    joshua-stauffer committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    c9d892a View commit details
    Browse the repository at this point in the history

  3. add generic partitioners

    @joshua-stauffer
    joshua-stauffer committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    3cc0639 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. move Partitioner out of typechecking

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    36239d9 View commit details
    Browse the repository at this point in the history

  2. update import

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    4854727 View commit details
    Browse the repository at this point in the history

  3. type

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    19164a2 View commit details
    Browse the repository at this point in the history

  4. schema update

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    8bf119b View commit details
    Browse the repository at this point in the history

  5. add partitioner to BatchRequest

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e695d19 View commit details
    Browse the repository at this point in the history

  6. update tests

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    a167db1 View commit details
    Browse the repository at this point in the history

  7. update test snapshot

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    50994e7 View commit details
    Browse the repository at this point in the history

  8. add partitioner to DataAsset.build_batch_request

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e20b62e View commit details
    Browse the repository at this point in the history

  9. update type stub

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    307263b View commit details
    Browse the repository at this point in the history

  10. update test snapshot

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5032352 View commit details
    Browse the repository at this point in the history

  11. missed one

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    8193139 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'develop' into f/v1-175/add_partitioners_to_batch_request

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    fa5cda8 View commit details
    Browse the repository at this point in the history

  13. add test

    @joshua-stauffer
    joshua-stauffer committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    9218fc9 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. add partitioner map to base class

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    f8780c2 View commit details
    Browse the repository at this point in the history

  2. add partitioner maps

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    c7e9103 View commit details
    Browse the repository at this point in the history

  3. rename spark partitioners

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    29e28af View commit details
    Browse the repository at this point in the history

  4. refactor sql partitioners

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    c0d2c1a View commit details
    Browse the repository at this point in the history

  5. rename spark partitioners

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    af5e0d9 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'develop' into f/v1-175/add_partitioners_to_batch_request

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    8795e5f View commit details
    Browse the repository at this point in the history

  7. schema update

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    9d2bdf5 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'f/v1-175/add_partitioners_to_batch_request' into f/v1-1… 

    …75/asset_uses_partitioner_from_batch_request
    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    f9b2210 View commit details
    Browse the repository at this point in the history

  9. update per rename

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    89f7193 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'develop' into f/v1-175/asset_uses_partitioner_from_batc… 

    …h_request
    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    4b405a0 View commit details
    Browse the repository at this point in the history

  11. implement method to replace batch_options property

    @joshua-stauffer
    joshua-stauffer committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    7bb9777 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. move partitioner resolve method to subclass

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    1c0c5d7 View commit details
    Browse the repository at this point in the history

  2. typeguard

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    baff01e View commit details
    Browse the repository at this point in the history

  3. add override

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    364dfaa View commit details
    Browse the repository at this point in the history

  4. rename param

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    f0d2f52 View commit details
    Browse the repository at this point in the history

  5. update types

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    d626182 View commit details
    Browse the repository at this point in the history

  6. remove from interface

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    a4e24b4 View commit details
    Browse the repository at this point in the history

  7. add types to subclass

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    269089e View commit details
    Browse the repository at this point in the history

  8. tweak type

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    f00a1fb View commit details
    Browse the repository at this point in the history

  9. revert

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    dbd95d0 View commit details
    Browse the repository at this point in the history

  10. types

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    1004b30 View commit details
    Browse the repository at this point in the history

  11. move type out of typechecking

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    8623707 View commit details
    Browse the repository at this point in the history

  12. move import

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    ac3ee51 View commit details
    Browse the repository at this point in the history

  13. try type as function

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    85f072a View commit details
    Browse the repository at this point in the history

  14. revert

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    9ed5cee View commit details
    Browse the repository at this point in the history

  15. hacky type workaround

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    c2ba4c7 View commit details
    Browse the repository at this point in the history

  16. use Type

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    fbf7657 View commit details
    Browse the repository at this point in the history

  17. spark: use batch request partitioner

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    abc76d5 View commit details
    Browse the repository at this point in the history

  18. sql: use batch request partitioner

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    0cf9e3a View commit details
    Browse the repository at this point in the history

  19. add sqlite partitioners

    @joshua-stauffer
    joshua-stauffer committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    de6fc0c View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. fix sqlite tests

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    454aef8 View commit details
    Browse the repository at this point in the history

  2. use get_batch_request_options instead of batch_request_options property

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    7a9fafa View commit details
    Browse the repository at this point in the history

  3. wip

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    6edf63f View commit details
    Browse the repository at this point in the history

  4. rename method

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    7da1552 View commit details
    Browse the repository at this point in the history

  5. sqlite tests passing

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    aa244ab View commit details
    Browse the repository at this point in the history

  6. update signature

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    d0b3961 View commit details
    Browse the repository at this point in the history

  7. update types

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    ab1149f View commit details
    Browse the repository at this point in the history

  8. move ConvertedDatetime partitioner into sql

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    e2e4d16 View commit details
    Browse the repository at this point in the history

  9. ensure sqlite assets have access to correct partitioner map

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    d801d17 View commit details
    Browse the repository at this point in the history

  10. schema update

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    f97c441 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'develop' into f/v1-175/asset_uses_partitioner_from_batc… 

    …h_request
    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    d88a277 View commit details
    Browse the repository at this point in the history

  12. schema update

    @joshua-stauffer
    joshua-stauffer committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    d0ff1ca View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. refactor partitioner to batch request in conftest

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    b924f97 View commit details
    Browse the repository at this point in the history

  2. update integration tests to use partitioner in batch request

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    7b7247f View commit details
    Browse the repository at this point in the history

  3. remove deprecated partitioner

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    3362915 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'develop' into f/v1-175/asset_uses_partitioner_from_batc… 

    …h_request
    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    7611ecb View commit details
    Browse the repository at this point in the history

  5. schema update

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    ff5abdf View commit details
    Browse the repository at this point in the history

  6. update integration tests

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    6137d92 View commit details
    Browse the repository at this point in the history

  7. update viral snippet

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    1ae3497 View commit details
    Browse the repository at this point in the history

  8. update tests related to fixture

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    d91e36d View commit details
    Browse the repository at this point in the history

  9. update postgresql tests

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    6c964ad View commit details
    Browse the repository at this point in the history

  10. update snippets

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    e7d28e3 View commit details
    Browse the repository at this point in the history

  11. public api

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    175814b View commit details
    Browse the repository at this point in the history

  12. schema update

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    53d8c89 View commit details
    Browse the repository at this point in the history

  13. update api excludes

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    4d3b6ee View commit details
    Browse the repository at this point in the history

  14. fix doc snippet

    @joshua-stauffer
    joshua-stauffer committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    4eab300 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. remove assert

    @joshua-stauffer
    joshua-stauffer committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    4b0651e View commit details
    Browse the repository at this point in the history

  2. remove comment

    @joshua-stauffer
    joshua-stauffer committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    43888ed View commit details
    Browse the repository at this point in the history

  3. remove batch config option keys method from batch config

    @joshua-stauffer
    joshua-stauffer committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    9d42d57 View commit details
    Browse the repository at this point in the history