Skip to content

Commit

Permalink
use pip to install tensorflow
Browse files Browse the repository at this point in the history
  • Loading branch information
@lhoestq
lhoestq committed Sep 25, 2020
1 parent f2d5d5e commit 58f4842
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion .circleci/config.yml
View file
Expand Up @@ -38,7 +38,7 @@ jobs:
steps:
- checkout
- run: conda install python=3.6 --yes
- run: conda install pytorch tensorflow>=2.2.0 --yes
- run: conda install pytorch --yes
- run: pip install virtualenv
- run: python -m virtualenv venv --system-site-packages
- run: "& venv/Scripts/activate.ps1"
Expand Down

1 comment on commit 58f4842

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Show benchmarks

PyArrow==0.17.1

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.019453 / 0.011353 (0.008100) 0.018200 / 0.011008 (0.007192) 0.045209 / 0.038508 (0.006701) 0.030645 / 0.023109 (0.007535) 0.247246 / 0.275898 (-0.028652) 0.254520 / 0.323480 (-0.068960) 0.010471 / 0.007986 (0.002486) 0.004046 / 0.004328 (-0.000283) 0.006993 / 0.004250 (0.002743) 0.050987 / 0.037052 (0.013935) 0.238359 / 0.258489 (-0.020130) 0.282572 / 0.293841 (-0.011269) 0.172486 / 0.128546 (0.043940) 0.139691 / 0.075646 (0.064044) 0.445375 / 0.419271 (0.026103) 0.544241 / 0.043533 (0.500708) 0.262241 / 0.255139 (0.007102) 0.268779 / 0.283200 (-0.014421) 0.090104 / 0.141683 (-0.051579) 1.812600 / 1.452155 (0.360446) 1.861285 / 1.492716 (0.368569)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.040352 / 0.037411 (0.002941) 0.020470 / 0.014526 (0.005944) 0.104646 / 0.176557 (-0.071911) 0.182648 / 0.737135 (-0.554487) 0.095910 / 0.296338 (-0.200428)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.206085 / 0.215209 (-0.009124) 2.268568 / 2.077655 (0.190914) 1.337652 / 1.504120 (-0.166468) 1.250967 / 1.541195 (-0.290227) 1.262769 / 1.468490 (-0.205721) 7.466295 / 4.584777 (2.881518) 6.335244 / 3.745712 (2.589532) 8.538097 / 5.269862 (3.268236) 7.697371 / 4.565676 (3.131694) 0.722343 / 0.424275 (0.298068) 0.011280 / 0.007607 (0.003673) 0.259049 / 0.226044 (0.033004) 2.528208 / 2.268929 (0.259279) 1.738146 / 55.444624 (-53.706479) 1.532588 / 6.876477 (-5.343889) 1.558222 / 2.142072 (-0.583850) 7.286850 / 4.805227 (2.481623) 7.418644 / 6.500664 (0.917980) 11.055561 / 0.075469 (10.980091)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 12.194166 / 1.841788 (10.352378) 15.144935 / 8.074308 (7.070627) 15.869216 / 10.191392 (5.677824) 0.516498 / 0.680424 (-0.163926) 0.299324 / 0.534201 (-0.234877) 0.859352 / 0.579283 (0.280069) 0.657474 / 0.434364 (0.223110) 0.827932 / 0.540337 (0.287595) 1.737670 / 1.386936 (0.350734)
PyArrow==1.0
Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric read_batch_formatted_as_numpy after write_array2d read_batch_formatted_as_numpy after write_flattened_sequence read_batch_formatted_as_numpy after write_nested_sequence read_batch_unformated after write_array2d read_batch_unformated after write_flattened_sequence read_batch_unformated after write_nested_sequence read_col_formatted_as_numpy after write_array2d read_col_formatted_as_numpy after write_flattened_sequence read_col_formatted_as_numpy after write_nested_sequence read_col_unformated after write_array2d read_col_unformated after write_flattened_sequence read_col_unformated after write_nested_sequence read_formatted_as_numpy after write_array2d read_formatted_as_numpy after write_flattened_sequence read_formatted_as_numpy after write_nested_sequence read_unformated after write_array2d read_unformated after write_flattened_sequence read_unformated after write_nested_sequence write_array2d write_flattened_sequence write_nested_sequence
new / old (diff) 0.020313 / 0.011353 (0.008960) 0.016473 / 0.011008 (0.005464) 0.058360 / 0.038508 (0.019852) 0.029094 / 0.023109 (0.005985) 0.336141 / 0.275898 (0.060243) 0.385937 / 0.323480 (0.062457) 0.008863 / 0.007986 (0.000878) 0.004692 / 0.004328 (0.000364) 0.008068 / 0.004250 (0.003817) 0.046243 / 0.037052 (0.009191) 0.336195 / 0.258489 (0.077706) 0.376740 / 0.293841 (0.082899) 0.165949 / 0.128546 (0.037403) 0.133741 / 0.075646 (0.058095) 0.449588 / 0.419271 (0.030317) 0.426417 / 0.043533 (0.382884) 0.334198 / 0.255139 (0.079059) 0.363355 / 0.283200 (0.080155) 0.094020 / 0.141683 (-0.047663) 1.817712 / 1.452155 (0.365557) 1.857914 / 1.492716 (0.365198)

Benchmark: benchmark_indices_mapping.json

metric select shard shuffle sort train_test_split
new / old (diff) 0.045895 / 0.037411 (0.008484) 0.022042 / 0.014526 (0.007516) 0.028105 / 0.176557 (-0.148451) 0.087030 / 0.737135 (-0.650105) 0.029525 / 0.296338 (-0.266814)

Benchmark: benchmark_iterating.json

metric read 5000 read 50000 read_batch 50000 10 read_batch 50000 100 read_batch 50000 1000 read_formatted numpy 5000 read_formatted pandas 5000 read_formatted tensorflow 5000 read_formatted torch 5000 read_formatted_batch numpy 5000 10 read_formatted_batch numpy 5000 1000 shuffled read 5000 shuffled read 50000 shuffled read_batch 50000 10 shuffled read_batch 50000 100 shuffled read_batch 50000 1000 shuffled read_formatted numpy 5000 shuffled read_formatted_batch numpy 5000 10 shuffled read_formatted_batch numpy 5000 1000
new / old (diff) 0.285269 / 0.215209 (0.070060) 2.684857 / 2.077655 (0.607202) 1.960608 / 1.504120 (0.456489) 1.807846 / 1.541195 (0.266652) 1.852637 / 1.468490 (0.384147) 7.389472 / 4.584777 (2.804695) 6.340022 / 3.745712 (2.594310) 8.779445 / 5.269862 (3.509583) 7.553453 / 4.565676 (2.987777) 0.743374 / 0.424275 (0.319099) 0.012734 / 0.007607 (0.005127) 0.318698 / 0.226044 (0.092654) 3.243249 / 2.268929 (0.974321) 15.218166 / 55.444624 (-40.226458) 3.785354 / 6.876477 (-3.091122) 2.165389 / 2.142072 (0.023316) 7.225433 / 4.805227 (2.420205) 2.277296 / 6.500664 (-4.223368) 0.035469 / 0.075469 (-0.040001)

Benchmark: benchmark_map_filter.json

metric filter map fast-tokenizer batched map identity map identity batched map no-op batched map no-op batched numpy map no-op batched pandas map no-op batched pytorch map no-op batched tensorflow
new / old (diff) 12.199493 / 1.841788 (10.357705) 15.299636 / 8.074308 (7.225328) 16.732932 / 10.191392 (6.541540) 0.923964 / 0.680424 (0.243540) 0.591616 / 0.534201 (0.057415) 0.895607 / 0.579283 (0.316324) 0.700595 / 0.434364 (0.266231) 0.881567 / 0.540337 (0.341229) 1.672533 / 1.386936 (0.285597)

Please sign in to comment.