Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL #34575

Closed
wants to merge 22 commits into from
Closed

[SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL #34575

wants to merge 22 commits into from

Commits on Nov 12, 2021

  1. first draft

    @Yaohua628
    Yaohua628 committed Nov 12, 2021
    Configuration menu
    Copy the full SHA
    dee06f6 View commit details
    Browse the repository at this point in the history

  2. nit of renaming file

    @Yaohua628
    Yaohua628 committed Nov 12, 2021
    Configuration menu
    Copy the full SHA
    06ac79e View commit details
    Browse the repository at this point in the history

  3. more meta

    @Yaohua628
    Yaohua628 committed Nov 12, 2021
    Configuration menu
    Copy the full SHA
    fc043fd View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2021

  1. metadata unapply

    @Yaohua628
    Yaohua628 committed Nov 15, 2021
    Configuration menu
    Copy the full SHA
    170378b View commit details
    Browse the repository at this point in the history

  2. nits

    @Yaohua628
    Yaohua628 committed Nov 15, 2021
    Configuration menu
    Copy the full SHA
    73593c5 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2021

  1. exhausive pattern match

    @Yaohua628
    Yaohua628 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    c531300 View commit details
    Browse the repository at this point in the history

  2. nit refactor

    @Yaohua628
    Yaohua628 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    bd28eb7 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2021

  1. addressed comments

    @Yaohua628
    Yaohua628 committed Nov 22, 2021
    Configuration menu
    Copy the full SHA
    e872d1f View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2021

  1. some nits comments

    @Yaohua628
    Yaohua628 committed Nov 25, 2021
    Configuration menu
    Copy the full SHA
    60bdbc5 View commit details
    Browse the repository at this point in the history

  2. simplify row cache

    @Yaohua628
    Yaohua628 committed Nov 25, 2021
    Configuration menu
    Copy the full SHA
    f78fe92 View commit details
    Browse the repository at this point in the history

  3. add todo

    @Yaohua628
    Yaohua628 committed Nov 25, 2021
    Configuration menu
    Copy the full SHA
    2baccdb View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2021

  1. comments and refactor test to use json

    @Yaohua628
    Yaohua628 committed Nov 26, 2021
    Configuration menu
    Copy the full SHA
    8b8b9fa View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2021

  1. flat and struct

    @Yaohua628
    Yaohua628 committed Dec 19, 2021
    Configuration menu
    Copy the full SHA
    d984b50 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. nit remove

    @Yaohua628
    Yaohua628 committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    0f6eccd View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2021

  1. comments

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    f780bf2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into spark-37273

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    a0a538c View commit details
    Browse the repository at this point in the history

  3. missing comments

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    3516e4e View commit details
    Browse the repository at this point in the history

  4. clean up with unsafe projection

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    00bda90 View commit details
    Browse the repository at this point in the history

  5. great cleanup

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    afa0a83 View commit details
    Browse the repository at this point in the history

  6. name nit

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    65e79ab View commit details
    Browse the repository at this point in the history

  7. change modification_time to TimestampType

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    3b3d635 View commit details
    Browse the repository at this point in the history

  8. 1000 in filescanrdd

    @Yaohua628
    Yaohua628 committed Dec 21, 2021
    Configuration menu
    Copy the full SHA
    4400f6a View commit details
    Browse the repository at this point in the history