Skip to content

Commit

Permalink
Fixed typo seperate->separate (#2292)
Browse files Browse the repository at this point in the history
  • Loading branch information
@laksh9950
laksh9950 committed Apr 30, 2021
1 parent 8afd0ba commit 8e9d869
Showing 0 changed files with 0 additions and 0 deletions.

1 comment on commit 8e9d869

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Show benchmarks

PyArrow==1.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.024260 / 0.011353 (0.012907) 0.015331 / 0.011008 (0.004323) 0.047466 / 0.038508 (0.008958) 0.035401 / 0.023109 (0.012292) 0.321302 / 0.275898 (0.045404) 0.362913 / 0.323480 (0.039433) 0.011299 / 0.007986 (0.003313) 0.004989 / 0.004328 (0.000660) 0.010734 / 0.004250 (0.006484) 0.044474 / 0.037052 (0.007421) 0.328721 / 0.258489 (0.070232) 0.364062 / 0.293841 (0.070221) 0.163914 / 0.128546 (0.035368) 0.125392 / 0.075646 (0.049746) 0.424722 / 0.419271 (0.005451) 0.640829 / 0.043533 (0.597297) 0.333334 / 0.255139 (0.078195) 0.354139 / 0.283200 (0.070939) 3.564865 / 0.141683 (3.423182) 1.748177 / 1.452155 (0.296022) 1.798078 / 1.492716 (0.305362)

Benchmark: benchmark_getitem_100B.json

metric get_batch_of_1024_random_rows get_batch_of_1024_rows get_first_row get_last_row
new / old (diff) 0.008342 / 0.018006 (-0.009664) 0.462857 / 0.000490 (0.462367) 0.001890 / 0.000200 (0.001690) 0.000080 / 0.000054 (0.000026)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.050042 / 0.037411 (0.012631) 0.024683 / 0.014526 (0.010157) 0.029594 / 0.176557 (-0.146962) 0.045860 / 0.737135 (-0.691275) 0.029461 / 0.296338 (-0.266877)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.484065 / 0.215209 (0.268856) 4.957734 / 2.077655 (2.880079) 2.227587 / 1.504120 (0.723467) 1.898347 / 1.541195 (0.357152) 1.905509 / 1.468490 (0.437018) 6.935652 / 4.584777 (2.350875) 6.179908 / 3.745712 (2.434196) 8.663452 / 5.269862 (3.393590) 7.494083 / 4.565676 (2.928407) 0.699947 / 0.424275 (0.275672) 0.010095 / 0.007607 (0.002488) 0.627573 / 0.226044 (0.401529) 6.311104 / 2.268929 (4.042176) 2.797158 / 55.444624 (-52.647466) 2.209393 / 6.876477 (-4.667084) 2.224380 / 2.142072 (0.082308) 7.179336 / 4.805227 (2.374109) 5.334798 / 6.500664 (-1.165866) 5.763550 / 0.075469 (5.688081)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 10.975696 / 1.841788 (9.133908) 12.984614 / 8.074308 (4.910306) 37.716692 / 10.191392 (27.525300) 0.861773 / 0.680424 (0.181349) 0.571674 / 0.534201 (0.037473) 0.776401 / 0.579283 (0.197118) 0.609027 / 0.434364 (0.174663) 0.709336 / 0.540337 (0.168998) 1.520349 / 1.386936 (0.133413)
PyArrow==latest
Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.022538 / 0.011353 (0.011185) 0.015226 / 0.011008 (0.004218) 0.048463 / 0.038508 (0.009955) 0.034812 / 0.023109 (0.011703) 0.345578 / 0.275898 (0.069680) 0.398159 / 0.323480 (0.074679) 0.010779 / 0.007986 (0.002794) 0.004665 / 0.004328 (0.000337) 0.010656 / 0.004250 (0.006405) 0.053374 / 0.037052 (0.016322) 0.344372 / 0.258489 (0.085883) 0.416638 / 0.293841 (0.122797) 0.161521 / 0.128546 (0.032975) 0.121906 / 0.075646 (0.046259) 0.410337 / 0.419271 (-0.008934) 0.384086 / 0.043533 (0.340554) 0.352693 / 0.255139 (0.097554) 0.376804 / 0.283200 (0.093605) 1.544434 / 0.141683 (1.402751) 1.745016 / 1.452155 (0.292862) 1.773263 / 1.492716 (0.280547)

Benchmark: benchmark_getitem_100B.json

metric get_batch_of_1024_random_rows get_batch_of_1024_rows get_first_row get_last_row
new / old (diff) 0.008315 / 0.018006 (-0.009691) 0.459075 / 0.000490 (0.458585) 0.002016 / 0.000200 (0.001816) 0.000084 / 0.000054 (0.000030)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.038501 / 0.037411 (0.001090) 0.024509 / 0.014526 (0.009983) 0.031467 / 0.176557 (-0.145089) 0.052109 / 0.737135 (-0.685026) 0.028962 / 0.296338 (-0.267376)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.437496 / 0.215209 (0.222287) 4.418609 / 2.077655 (2.340954) 1.980887 / 1.504120 (0.476767) 1.705962 / 1.541195 (0.164767) 1.737565 / 1.468490 (0.269075) 6.745663 / 4.584777 (2.160886) 5.948422 / 3.745712 (2.202710) 8.395193 / 5.269862 (3.125331) 7.508204 / 4.565676 (2.942527) 0.696995 / 0.424275 (0.272720) 0.010697 / 0.007607 (0.003090) 0.571976 / 0.226044 (0.345931) 5.749171 / 2.268929 (3.480242) 2.591316 / 55.444624 (-52.853308) 2.133412 / 6.876477 (-4.743065) 2.043760 / 2.142072 (-0.098313) 6.749740 / 4.805227 (1.944512) 4.957334 / 6.500664 (-1.543331) 5.537873 / 0.075469 (5.462404)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 10.588439 / 1.841788 (8.746651) 12.444115 / 8.074308 (4.369807) 36.194524 / 10.191392 (26.003132) 0.843749 / 0.680424 (0.163325) 0.569404 / 0.534201 (0.035203) 0.759999 / 0.579283 (0.180715) 0.605740 / 0.434364 (0.171376) 0.672877 / 0.540337 (0.132539) 1.461738 / 1.386936 (0.074802)

CML watermark

Please sign in to comment.