Performance Measurements with Google CS and_gsutil

Some performance results

Tested from Germany on June 20th 2019 from a Lenovo X270 Thinkpad. Internet connection was ADSL with 16 MBps downstream and 1 MBps upstream. The services (buckets) tested were located in the US.

Google Cloud Storage service

These options need to be modified inside the file ~/.boto in order to interact with Google Cloud Storage via the command line tool gsutil.

access_key = insert here your access key
secret_key = insert here your secret access key

10 Files, 1 Byte (2^0 Byte) to 16 MB (2^24 Byte) each, parallel transfer

The benchmark has ben started by this script:

for i in 512 1024 2048 4096 8192 16384 32768 65536 131072 262144 524288 1048576 2097152 4194304 8388608
do
    for x in 1 2 3 4 5
    do
        ./ossperf.sh -n 10 -s ${i} -b ossperf-testbucket -g -p -o 2>&1 
        # "$?" contains the return code of the last command executed.
        if [ "$?" -ne "0" ] ; then
            exit 1
        fi
        sleep 10
    done
done

This is the result.

DATE TIME NUM_FILES SIZE_FILES TIME_CREATE_BUCKET TIME_OBJECTS_UPLOAD TIME_OBJECTS_LIST TIME_OBJECTS_DOWNLOAD TIME_ERASE_OBJECTS TIME_ERASE_BUCKET TIME_SUM BANDWIDTH_OBJECTS_UPLOAD BANDWIDTH_OBJECTS_DOWNLOAD
2019-06-20 19:45:47 10 512 6.385 1.604 0.771 3.853 1.395 1.446 15.454 0.025 0.010
2019-06-20 19:46:20 10 512 6.161 1.544 0.825 3.507 1.490 1.395 14.922 0.026 0.011
2019-06-20 19:46:52 10 512 5.531 1.615 0.787 3.518 1.379 1.916 14.746 0.025 0.011
2019-06-20 19:47:26 10 512 5.834 1.832 0.752 3.218 1.464 2.160 15.260 0.022 0.012
2019-06-20 19:47:59 10 512 5.992 2.018 0.809 3.401 1.387 1.834 15.441 0.020 0.012
2019-06-20 19:48:33 10 1024 6.105 1.614 0.831 4.152 1.445 1.829 15.976 0.050 0.019
2019-06-20 19:49:06 10 1024 5.876 1.540 0.834 3.589 1.523 2.119 15.481 0.053 0.022
2019-06-20 19:49:40 10 1024 5.815 1.504 0.853 3.685 1.888 2.106 15.851 0.054 0.022
2019-06-20 19:50:12 10 1024 5.507 1.532 0.794 3.545 1.401 1.723 14.502 0.053 0.023
2019-06-20 19:50:45 10 1024 6.303 1.397 0.787 3.579 1.430 1.478 14.974 0.058 0.022
2019-06-20 19:51:18 10 2048 5.677 1.579 0.782 3.484 1.362 2.083 14.967 0.103 0.047
2019-06-20 19:51:52 10 2048 6.188 1.592 0.803 3.912 1.890 2.129 16.514 0.102 0.041
2019-06-20 19:52:26 10 2048 6.114 1.588 0.845 3.972 1.410 1.811 15.740 0.103 0.041
2019-06-20 19:52:59 10 2048 5.653 1.603 0.745 3.511 1.424 2.104 15.040 0.102 0.046
2019-06-20 19:53:33 10 2048 6.656 1.644 0.832 3.721 1.408 1.793 16.054 0.099 0.044
2019-06-20 19:54:05 10 4096 5.535 1.501 0.807 3.137 1.387 1.749 14.116 0.218 0.104
2019-06-20 19:54:38 10 4096 6.140 1.717 0.859 3.449 1.461 1.692 15.318 0.190 0.095
2019-06-20 19:55:11 10 4096 6.203 1.537 0.746 3.775 1.367 1.426 15.054 0.213 0.086
2019-06-20 19:55:43 10 4096 5.921 1.540 0.848 2.985 1.388 1.377 14.059 0.212 0.109
2019-06-20 19:56:15 10 4096 5.370 1.650 0.771 3.224 1.352 1.395 13.762 0.198 0.101
2019-06-20 19:56:48 10 8192 5.860 1.606 0.874 3.665 1.338 2.112 15.455 0.408 0.178
2019-06-20 19:57:21 10 8192 6.155 1.623 0.870 3.343 1.321 2.137 15.449 0.403 0.196
2019-06-20 19:57:55 10 8192 5.751 1.753 0.796 4.055 1.380 1.444 15.179 0.373 0.161
2019-06-20 19:58:27 10 8192 5.896 1.642 0.807 3.500 1.371 1.367 14.583 0.399 0.187
2019-06-20 19:59:00 10 8192 5.968 1.679 0.746 3.281 1.181 1.460 14.315 0.390 0.199
2019-06-20 19:59:34 10 16384 6.170 1.981 0.767 3.209 1.384 2.157 15.668 0.661 0.408
2019-06-20 20:00:07 10 16384 5.313 1.822 0.854 3.872 1.435 1.404 14.700 0.719 0.338
2019-06-20 20:00:42 10 16384 6.753 1.827 0.864 4.056 1.374 2.228 17.102 0.717 0.323
2019-06-20 20:01:16 10 16384 5.670 2.223 0.813 3.688 1.450 1.723 15.567 0.589 0.355
2019-06-20 20:01:50 10 16384 6.174 1.872 0.783 3.436 1.391 2.409 16.065 0.700 0.381
2019-06-20 20:02:25 10 32768 5.817 3.009 0.795 3.347 1.439 1.479 15.886 0.871 0.783
2019-06-20 20:02:59 10 32768 5.837 2.270 0.833 3.323 1.309 2.228 15.800 1.154 0.788
2019-06-20 20:03:34 10 32768 6.066 2.331 0.864 3.775 1.431 1.818 16.285 1.124 0.694
2019-06-20 20:04:09 10 32768 6.388 2.428 0.771 3.608 1.447 1.668 16.310 1.079 0.726
2019-06-20 20:04:44 10 32768 6.452 2.505 0.825 3.580 1.399 1.913 16.674 1.046 0.732
2019-06-20 20:05:20 10 65536 6.091 2.721 0.829 3.733 1.420 1.511 16.305 1.926 1.404
2019-06-20 20:05:57 10 65536 5.847 2.763 0.986 4.433 1.400 2.070 17.499 1.897 1.182
2019-06-20 20:06:31 10 65536 5.641 2.655 0.775 3.374 1.362 1.270 15.077 1.974 1.553
2019-06-20 20:07:06 10 65536 5.720 2.589 0.815 3.288 1.376 1.377 15.165 2.025 1.594
2019-06-20 20:07:42 10 65536 5.831 2.500 0.871 3.356 1.308 2.065 15.931 2.097 1.562
2019-06-20 20:08:19 10 131072 5.050 3.900 0.725 3.592 1.411 1.358 16.036 2.688 2.919
2019-06-20 20:08:57 10 131072 5.975 3.975 0.754 3.491 1.453 1.315 16.963 2.637 3.003
2019-06-20 20:09:36 10 131072 6.204 3.864 0.859 3.525 1.336 2.159 17.947 2.713 2.974
2019-06-20 20:10:14 10 131072 5.715 4.031 0.864 3.527 1.262 1.418 16.817 2.601 2.972
2019-06-20 20:10:53 10 131072 6.103 3.910 0.766 3.605 1.373 1.731 17.488 2.681 2.908
2019-06-20 20:11:38 10 262144 5.359 6.371 0.812 3.891 1.440 2.070 19.943 3.291 5.389
2019-06-20 20:12:22 10 262144 5.748 6.205 0.756 3.588 1.374 1.610 19.281 3.379 5.844
2019-06-20 20:13:05 10 262144 5.625 6.184 0.772 3.595 1.294 1.271 18.741 3.391 5.833
2019-06-20 20:13:49 10 262144 5.168 6.173 0.739 4.091 1.418 1.753 19.342 3.397 5.126
2019-06-20 20:14:33 10 262144 5.219 6.398 0.783 4.071 1.407 1.890 19.768 3.277 5.151
2019-06-20 20:15:31 10 524288 5.786 11.431 0.775 4.480 1.290 1.778 25.540 3.669 9.362
2019-06-20 20:16:27 10 524288 6.312 11.188 0.697 4.297 1.203 1.794 25.491 3.748 9.761
2019-06-20 20:17:24 10 524288 5.266 11.096 0.818 4.976 1.306 1.770 25.232 3.780 8.429
2019-06-20 20:18:25 10 524288 6.241 11.276 0.896 4.836 3.560 2.113 28.922 3.719 8.673
2019-06-20 20:19:24 10 524288 5.656 10.831 0.914 4.024 1.331 1.796 24.552 3.872 10.423
2019-06-20 20:20:52 10 1048576 5.696 21.354 0.899 6.889 1.539 1.832 38.209 3.928 12.176
2019-06-20 20:22:23 10 1048576 6.761 21.839 0.783 6.810 1.504 2.391 40.088 3.841 12.318
2019-06-20 20:23:53 10 1048576 6.213 21.037 0.794 5.690 2.007 1.935 37.676 3.987 14.742
2019-06-20 20:25:20 10 1048576 5.877 20.965 0.780 5.566 1.379 1.689 36.256 4.001 15.071
2019-06-20 20:26:50 10 1048576 6.510 21.703 0.739 6.084 1.363 2.189 38.588 3.865 13.787
2019-06-20 22:09:29 10 1048576 6.584 20.886 0.839 5.571 1.516 1.732 37.128 4.016 15.057
2019-06-20 22:10:52 10 1048576 5.271 21.053 0.912 5.916 2.239 1.860 37.251 3.984 14.179
2019-06-20 22:12:19 10 1048576 6.399 20.911 0.877 6.858 1.654 2.640 39.339 4.011 12.231
2019-06-20 22:13:41 10 1048576 5.741 20.786 0.755 5.418 1.390 1.351 35.441 4.035 15.482
2019-06-20 22:15:06 10 1048576 6.301 22.061 0.786 5.702 1.419 2.153 38.422 3.802 14.711
2019-06-20 22:17:25 10 2097152 5.968 40.397 0.776 8.934 0.990 1.317 58.382 4.153 18.779
2019-06-20 22:19:46 10 2097152 6.001 40.107 0.602 12.748 1.007 1.412 61.877 4.183 13.160
2019-06-20 22:22:06 10 2097152 6.284 40.983 0.727 10.553 0.861 0.963 60.371 4.093 15.898
2019-06-20 22:24:26 10 2097152 5.830 39.776 0.624 11.065 0.989 2.234 60.518 4.217 15.162
2019-06-20 22:26:48 10 2097152 6.187 40.682 0.813 9.424 0.895 1.765 59.766 4.123 17.802
2019-06-20 22:31:06 10 4194304 7.688 78.711 0.669 17.425 0.998 1.512 107.003 4.262 19.256
2019-06-20 22:35:24 10 4194304 6.013 79.363 0.647 16.944 2.579 1.420 106.966 4.227 19.803