Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-1990: [JS] C++ Refactor, Add DataFrame #1482

Closed
wants to merge 75 commits into from
Closed

ARROW-1990: [JS] C++ Refactor, Add DataFrame #1482

wants to merge 75 commits into from

Commits on Jan 11, 2018

  1. fix typings issues (ARROW-1903)

    @trxcllnt
    trxcllnt committed Jan 11, 2018
    Configuration menu
    Copy the full SHA
    74e828a View commit details
    Browse the repository at this point in the history

  2. ship es2015 commonJS in main package to avoid confusion

    @trxcllnt
    trxcllnt committed Jan 11, 2018
    Configuration menu
    Copy the full SHA
    6ff18e9 View commit details
    Browse the repository at this point in the history

  3. update dependencies and add es6+ umd targets to jest transform ignore… 

    … patterns to fix ci
    @trxcllnt
    trxcllnt committed Jan 11, 2018
    Configuration menu
    Copy the full SHA
    62db338 View commit details
    Browse the repository at this point in the history

  4. WIP -- refactor types to closer match arrow-cpp

    @trxcllnt
    trxcllnt committed Jan 11, 2018
    Configuration menu
    Copy the full SHA
    61dc699 View commit details
    Browse the repository at this point in the history

  5. fix struct and map rows

    @trxcllnt
    trxcllnt committed Jan 11, 2018
    Configuration menu
    Copy the full SHA
    d18d915 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2018

  1. Add perf tests for table scans

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    a1edac2 View commit details
    Browse the repository at this point in the history

  2. Add basic DataFrame impl ... 

    ... and a bunch of performance tests for various scanning approaches
    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    30f0330 View commit details
    Browse the repository at this point in the history

  3. add DataFrame filter and count ops

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    796f45d View commit details
    Browse the repository at this point in the history

  4. Add concept of predicates for filtering dataframes

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    4d9e8c0 View commit details
    Browse the repository at this point in the history

  5. Add DictionaryVector optimization for equals predicate

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    aa999f8 View commit details
    Browse the repository at this point in the history

  6. Remove Chunked/Simple DataFrame distinction

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    2744c63 View commit details
    Browse the repository at this point in the history

  7. clean up table benchmarks

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    6a41d68 View commit details
    Browse the repository at this point in the history

  8. Refactor DataFrame to extend Vector<StructRow>

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    e8979ba View commit details
    Browse the repository at this point in the history

  9. Moved DataFrame ops to Table. DataFrame is now an interface

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    1d60aa1 View commit details
    Browse the repository at this point in the history

  10. Move Table out of the Vector hierarchy

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    a9fff89 View commit details
    Browse the repository at this point in the history

  11. Cleanup

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    a788db3 View commit details
    Browse the repository at this point in the history

  12. linter

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    2e118ab View commit details
    Browse the repository at this point in the history

  13. Minor tweaks

    Brian Hulette committed Jan 12, 2018
    Configuration menu
    Copy the full SHA
    2f4a349 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2018

  1. Add DataFrame.countBy operation

    Brian Hulette committed Jan 15, 2018
    Configuration menu
    Copy the full SHA
    6719147 View commit details
    Browse the repository at this point in the history

  2. Add table unit tests... 

    .. also found and resolved some minor bugs (get(idx) batch length check should
be <=, various extern issues with UMD builds)
    Brian Hulette committed Jan 15, 2018
    Configuration menu
    Copy the full SHA
    7244887 View commit details
    Browse the repository at this point in the history

  3. Fixed countBy(string)

    Brian Hulette committed Jan 15, 2018
    Configuration menu
    Copy the full SHA
    20717d5 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2018

  1. cleanup

    Brian Hulette committed Jan 16, 2018
    Configuration menu
    Copy the full SHA
    edcbdbe View commit details
    Browse the repository at this point in the history

  2. Add license headers

    Brian Hulette committed Jan 16, 2018
    Configuration menu
    Copy the full SHA
    e20decd View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2018

  1. passing all tests except es5 umd mangler ones

    @trxcllnt
    trxcllnt committed Jan 18, 2018
    Configuration menu
    Copy the full SHA
    fa7c17a View commit details
    Browse the repository at this point in the history

  2. fix rest of the mangling issues

    @trxcllnt
    trxcllnt committed Jan 18, 2018
    Configuration menu
    Copy the full SHA
    e3f629d View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2018

  1. rename table.ts to recordbatch.ts in preparation for merging latest

    @trxcllnt
    trxcllnt committed Jan 19, 2018
    Configuration menu
    Copy the full SHA
    f3f3b86 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'ccri/table-scan-perf' into js-cpp-refac… 

    …tor-merge_with-table-scan-perf
    @trxcllnt
    trxcllnt committed Jan 19, 2018
    Configuration menu
    Copy the full SHA
    d2b18d5 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of github.com:apache/arrow into js-cpp-refactor… 

    …-merge_with-table-scan-perf
    @trxcllnt
    trxcllnt committed Jan 19, 2018
    Configuration menu
    Copy the full SHA
    6c91ed4 View commit details
    Browse the repository at this point in the history

  4. Rename asJSON to toJSON

    Brian Hulette committed Jan 19, 2018
    Configuration menu
    Copy the full SHA
    e761eee View commit details
    Browse the repository at this point in the history

  5. Don't recompute total length

    Brian Hulette committed Jan 19, 2018
    Configuration menu
    Copy the full SHA
    0126dc4 View commit details
    Browse the repository at this point in the history

  6. use Math.fround

    Brian Hulette committed Jan 19, 2018
    Configuration menu
    Copy the full SHA
    0620cfd View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2018

  1. fix package.json bin entry

    @trxcllnt
    trxcllnt committed Jan 20, 2018
    Configuration menu
    Copy the full SHA
    e859e13 View commit details
    Browse the repository at this point in the history

  2. export visitors

    @trxcllnt
    trxcllnt committed Jan 20, 2018
    Configuration menu
    Copy the full SHA
    700a47c View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2018

  1. export vector views, allow cloning data as another type

    @trxcllnt
    trxcllnt committed Jan 22, 2018
    Configuration menu
    Copy the full SHA
    e81082f View commit details
    Browse the repository at this point in the history

  2. rename numRows to length, add table.getColumn()

    @trxcllnt
    trxcllnt committed Jan 22, 2018
    Configuration menu
    Copy the full SHA
    b7f5bfb View commit details
    Browse the repository at this point in the history

  3. Merge branch 'table-scan-perf' of github.com:ccri/arrow into js-cpp-r… 

    …efactor
    @trxcllnt
    trxcllnt committed Jan 22, 2018
    Configuration menu
    Copy the full SHA
    87334a5 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2018

  1. add asEpochMs to date and timestamp vectors

    @trxcllnt
    trxcllnt committed Jan 23, 2018
    Configuration menu
    Copy the full SHA
    614b688 View commit details
    Browse the repository at this point in the history

  2. Don't read OFFSET vector for FixedSizeList

    Brian Hulette committed Jan 23, 2018
    Configuration menu
    Copy the full SHA
    5bb63af View commit details
    Browse the repository at this point in the history

  3. Merge pull request #2 from ccri/fixed-size-list 

    Don't read OFFSET vector for FixedSizeList
    @trxcllnt
    trxcllnt committed Jan 23, 2018
    Configuration menu
    Copy the full SHA
    e33c068 View commit details
    Browse the repository at this point in the history

  4. use the dictionary of the last chunked vector list for chunked dictio… 

    …nary vectors
    @trxcllnt
    trxcllnt committed Jan 23, 2018
    Configuration menu
    Copy the full SHA
    c0fd2f9 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2018

  1. make it easier to run all integration tests from local data

    @trxcllnt
    trxcllnt committed Jan 24, 2018
    Configuration menu
    Copy the full SHA
    e537789 View commit details
    Browse the repository at this point in the history

  2. slice the flat data values before returning an iterator of them

    @trxcllnt
    trxcllnt committed Jan 24, 2018
    Configuration menu
    Copy the full SHA
    fe31ee0 View commit details
    Browse the repository at this point in the history

  3. run integration tests with local data for coverage stats

    @trxcllnt
    trxcllnt committed Jan 24, 2018
    Configuration menu
    Copy the full SHA
    40b3638 View commit details
    Browse the repository at this point in the history

  4. lazily allocate table and recordbatch columns, support NestedView's g… 

    …etChildAt(i) method in ChunkedView
    @trxcllnt
    trxcllnt committed Jan 24, 2018
    Configuration menu
    Copy the full SHA
    54d4f5b View commit details
    Browse the repository at this point in the history

  5. Fix perf

    Brian Hulette committed Jan 24, 2018
    Configuration menu
    Copy the full SHA
    a00415e View commit details
    Browse the repository at this point in the history

  6. Add Table.fromStruct

    Brian Hulette committed Jan 24, 2018
    Configuration menu
    Copy the full SHA
    c8cd286 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2018

  1. Merge pull request #3 from ccri/table-from-struct 

    Table from struct
    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    a5f200f View commit details
    Browse the repository at this point in the history

  2. add test:integration npm script

    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    7e43b78 View commit details
    Browse the repository at this point in the history

  3. rename ChunkData's fields so it's more clear they're not semantically… 

    … similar to other similarly named fields
    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    18807c6 View commit details
    Browse the repository at this point in the history

  4. compute chunked nested childData list correctly

    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    f1dead0 View commit details
    Browse the repository at this point in the history

  5. check bounds in getChildAt(i) to avoid NPEs

    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    8ddce0a View commit details
    Browse the repository at this point in the history

  6. Fix exception for empty Table

    Brian Hulette committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    7bc7363 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #4 from ccri/empty-table 

    Fix exception for empty Table
    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    272d293 View commit details
    Browse the repository at this point in the history

  8. Merge pull request #1 from trxcllnt/js-cpp-refactor 

    Refactor JS implementation to more closely resemble C++
    @TheNeuralBit
    TheNeuralBit committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    016ba78 View commit details
    Browse the repository at this point in the history

  9. fix vector perf tests

    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    9769773 View commit details
    Browse the repository at this point in the history

  10. fix lint

    @trxcllnt
    trxcllnt committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    f3cde1a View commit details
    Browse the repository at this point in the history

  11. Merge pull request #2 from trxcllnt/js-cpp-refactor 

    fix perf tests + lint
    @TheNeuralBit
    TheNeuralBit committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    579ab1f View commit details
    Browse the repository at this point in the history

  12. Create predicate namespace

    Brian Hulette committed Jan 25, 2018
    Configuration menu
    Copy the full SHA
    25e6af7 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2018

  1. add more view, predicate externs

    @trxcllnt
    trxcllnt committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    dc7c728 View commit details
    Browse the repository at this point in the history

  2. add src/predicate to the list of exports we should save from uglify

    @trxcllnt
    trxcllnt committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    25cdc4a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'extern-woes' into js-cpp-refactor

    @trxcllnt
    trxcllnt committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    e148ee4 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'table-scan-perf' of github.com:ccri/arrow into js-cpp-r… 

    …efactor
    @trxcllnt
    trxcllnt committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    f7bb0ed View commit details
    Browse the repository at this point in the history

  5. Create predicate namespace

    Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    f6adfb3 View commit details
    Browse the repository at this point in the history

  6. add more view, predicate externs

    @trxcllnt
    trxcllnt authored and Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    5a91fab View commit details
    Browse the repository at this point in the history

  7. add src/predicate to the list of exports we should save from uglify

    @trxcllnt
    trxcllnt authored and Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    4a41b18 View commit details
    Browse the repository at this point in the history

  8. Merge remote-tracking branch 'origin/master' into table-scan-perf

    Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    8cf2473 View commit details
    Browse the repository at this point in the history

  9. Fix perf

    Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    5bdf17f View commit details
    Browse the repository at this point in the history

  10. Add optional bind callback to scan

    Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    1910962 View commit details
    Browse the repository at this point in the history

  11. Add more Table unit tests

    Brian Hulette committed Jan 26, 2018
    Configuration menu
    Copy the full SHA
    dbe7f81 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2018

  1. Merge branch 'table-scan-perf' of github.com:ccri/arrow into js-cpp-r… 

    …efactor
    @trxcllnt
    trxcllnt committed Jan 27, 2018
    Configuration menu
    Copy the full SHA
    10c48ad View commit details
    Browse the repository at this point in the history

  2. fix more externs

    @trxcllnt
    trxcllnt committed Jan 27, 2018
    Configuration menu
    Copy the full SHA
    3d5240a View commit details
    Browse the repository at this point in the history

  3. fix closure es5/umd toString() iterator

    @trxcllnt
    trxcllnt committed Jan 27, 2018
    Configuration menu
    Copy the full SHA
    fe300df View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2018

  1. Merge pull request #4 from trxcllnt/js-cpp-refactor 

    fix closure es5/umd toString() iterator
    @TheNeuralBit
    TheNeuralBit committed Jan 29, 2018
    Configuration menu
    Copy the full SHA
    16b9ccb View commit details
    Browse the repository at this point in the history

  2. even more table tests

    Brian Hulette committed Jan 29, 2018
    Configuration menu
    Copy the full SHA
    04b1838 View commit details
    Browse the repository at this point in the history

  3. <, > are not commutative, misc cleanup

    Brian Hulette committed Jan 29, 2018
    Configuration menu
    Copy the full SHA
    52f1e0e View commit details
    Browse the repository at this point in the history