Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Schema overhaul #785

Merged
merged 44 commits into from
Jan 13, 2022
Merged

Schema overhaul #785

merged 44 commits into from
Jan 13, 2022

Commits on Dec 10, 2021

  1. Schema overhaul 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    8e97b86 View commit details
    Browse the repository at this point in the history

  2. Updated idl 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    c214013 View commit details
    Browse the repository at this point in the history

  3. Renamed FlyteDataset to StructuredDataset 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    a4cfe55 View commit details
    Browse the repository at this point in the history

  4. Fix test 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    e3bc1f2 View commit details
    Browse the repository at this point in the history

  5. Address comments 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    61ebbb8 View commit details
    Browse the repository at this point in the history

  6. Fixed lint 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    7078745 View commit details
    Browse the repository at this point in the history

  7. Fixed lint 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    2420fe2 View commit details
    Browse the repository at this point in the history

  8. Updated get literal type 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    9734284 View commit details
    Browse the repository at this point in the history

  9. Updated model file 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    fccd506 View commit details
    Browse the repository at this point in the history

  10. test 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    90cc7cb View commit details
    Browse the repository at this point in the history

  11. Added tests 

    Signed-off-by: Kevin <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    408d3f8 View commit details
    Browse the repository at this point in the history

  12. Fixed test 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    0bc5b7a View commit details
    Browse the repository at this point in the history

  13. Fixed test 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    327c80f View commit details
    Browse the repository at this point in the history

  14. Fixed test 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    90f831f View commit details
    Browse the repository at this point in the history

  15. updated interfaces 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    f88f32d View commit details
    Browse the repository at this point in the history

  16. add combined outline, allow not implemented error 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    85d48c1 View commit details
    Browse the repository at this point in the history

  17. new interface, partially thought through 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    8926beb View commit details
    Browse the repository at this point in the history

  18. to literal 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    8370ead View commit details
    Browse the repository at this point in the history

  19. to python value 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    5b277f5 View commit details
    Browse the repository at this point in the history

  20. add format 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    9aa9b48 View commit details
    Browse the repository at this point in the history

  21. register and comments 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor @pingsutw
    wild-endeavor authored and pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    a140bda View commit details
    Browse the repository at this point in the history

  22. Init new schema api 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    9dfbadf View commit details
    Browse the repository at this point in the history

  23. Fixed tests 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    9a54fbb View commit details
    Browse the repository at this point in the history

  24. Fixed tests 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    6ca053c View commit details
    Browse the repository at this point in the history

  25. Fixed tests 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    56d68f6 View commit details
    Browse the repository at this point in the history

  26. Add tests 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    8d024fb View commit details
    Browse the repository at this point in the history

  27. Add spark handlers 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 10, 2021
    Configuration menu
    Copy the full SHA
    0c069f1 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. Updated idl version 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    521dcec View commit details
    Browse the repository at this point in the history

  2. Updated idl version 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    363712d View commit details
    Browse the repository at this point in the history

  3. update comments 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    f48ed5e View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2021

  1. [pr into #785] (#793) 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
    @wild-endeavor
    wild-endeavor committed Dec 29, 2021
    Configuration menu
    Copy the full SHA
    2033dc8 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into structured-dataset-… 

    …proposal

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Dec 29, 2021
    Configuration menu
    Copy the full SHA
    6600e35 View commit details
    Browse the repository at this point in the history

  3. [pr into #785] Clean up is_remote calls (#795) 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Dec 29, 2021
    Configuration menu
    Copy the full SHA
    99398ec View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2022

  1. [pr into #785] wip - Compatibility testing (#796) 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    15e70b4 View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2022

  1. [pr into #785] Format cleanup (#803) 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Jan 7, 2022
    Configuration menu
    Copy the full SHA
    4f42207 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2022

  1. [pr into #785] Turn structured dataset into dataclass (#802) 

    * [pr into #785] Turn structured dataset into dataclass

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Inspect class

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Inspect class

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Resolved conflict

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Resolved conflict

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Resolved conflict

Signed-off-by: Kevin Su <pingsutw@apache.org>

* update

Signed-off-by: Kevin Su <pingsutw@apache.org>

* tests

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* nit

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* improve printing

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* Fixed tests and lint

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Fixed tests

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Fixed tests

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Pandas read local dir instead of file

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Remove duplicate code

Signed-off-by: Kevin Su <pingsutw@apache.org>

* support FlyteSchema -> StructuredDataset

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Remove inspect.isclass

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Fix tests

Signed-off-by: Kevin Su <pingsutw@apache.org>

* Fix lint

Signed-off-by: Kevin Su <pingsutw@apache.org>

Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @pingsutw @wild-endeavor
    pingsutw and wild-endeavor committed Jan 11, 2022
    Configuration menu
    Copy the full SHA
    e66f80f View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2022

  1. [pr into #785] testing (#812) 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    884a440 View commit details
    Browse the repository at this point in the history

  2. [pr into #785] Remove fsspec (#811) 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    f9358c8 View commit details
    Browse the repository at this point in the history

  3. minor lint 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    021992d View commit details
    Browse the repository at this point in the history

  4. More tests 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    1870c7f View commit details
    Browse the repository at this point in the history

  5. remove iter 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    02181b5 View commit details
    Browse the repository at this point in the history

  6. more tests 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    12f6c7d View commit details
    Browse the repository at this point in the history

  7. fix test 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
    @wild-endeavor
    wild-endeavor committed Jan 12, 2022
    Configuration menu
    Copy the full SHA
    66026b3 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2022

  1. Resolve conflict 

    Signed-off-by: Kevin Su <pingsutw@apache.org>
    @pingsutw
    pingsutw committed Jan 13, 2022
    Configuration menu
    Copy the full SHA
    bbf5b88 View commit details
    Browse the repository at this point in the history