Tracking issue of RFC-0191: Async Streaming IO

[Xuanwo]

Tracking of RFC-0191: Async Streaming IO

Checklist

• refactor: Make read return BytesStream instead #192
• refactor: Make write return BytesSink instead #194
• feat(io_util): Implement observer for sink and stream #198
• feat: Implement io_util like into_sink and into_stream #197
• add docs for stream and sink

[Xuanwo]

Demo benchmark:

read:

read_full/4.00 KiB      time:   [455.70 us 466.18 us 476.93 us]
                        thrpt:  [8.1904 MiB/s 8.3794 MiB/s 8.5719 MiB/s]
read_full/256 KiB       time:   [530.63 us 544.30 us 557.84 us]
                        thrpt:  [448.16 MiB/s 459.30 MiB/s 471.14 MiB/s]
read_full/4.00 MiB      time:   [1.5569 ms 1.6152 ms 1.6743 ms]
                        thrpt:  [2.3330 GiB/s 2.4184 GiB/s 2.5090 GiB/s]
read_full/16.0 MiB      time:   [5.7337 ms 5.9087 ms 6.0813 ms]
                        thrpt:  [2.5693 GiB/s 2.6444 GiB/s 2.7251 GiB/s]

read2:

read_full/4.00 KiB      time:   [455.67 us 466.03 us 476.21 us]
                        thrpt:  [8.2027 MiB/s 8.3819 MiB/s 8.5725 MiB/s]
                 change:
                        time:   [-2.1168% +0.6241% +3.8735%] (p = 0.68 > 0.05)
                        thrpt:  [-3.7291% -0.6203% +2.1625%]
                        No change in performance detected.
read_full/256 KiB       time:   [521.04 us 535.20 us 548.74 us]
                        thrpt:  [455.59 MiB/s 467.11 MiB/s 479.81 MiB/s]
                 change:
                        time:   [-7.8470% -4.7987% -1.4955%] (p = 0.01 < 0.05)
                        thrpt:  [+1.5182% +5.0406% +8.5152%]
                        Performance has improved.
read_full/4.00 MiB      time:   [1.4571 ms 1.5184 ms 1.5843 ms]
                        thrpt:  [2.4655 GiB/s 2.5725 GiB/s 2.6808 GiB/s]
                 change:
                        time:   [-5.4403% -1.5696% +2.3719%] (p = 0.44 > 0.05)
                        thrpt:  [-2.3170% +1.5946% +5.7533%]
                        No change in performance detected.
read_full/16.0 MiB      time:   [5.0201 ms 5.2105 ms 5.3986 ms]
                        thrpt:  [2.8943 GiB/s 2.9988 GiB/s 3.1125 GiB/s]
                 change:
                        time:   [-15.917% -11.816% -7.5219%] (p = 0.00 < 0.05)
                        thrpt:  [+8.1337% +13.400% +18.930%]
                        Performance has improved.

read2 powered Reader

read_full/4.00 KiB      time:   [482.04 us 493.32 us 504.34 us]
                        thrpt:  [7.7453 MiB/s 7.9183 MiB/s 8.1037 MiB/s]
read_full/256 KiB       time:   [549.86 us 561.84 us 574.56 us]
                        thrpt:  [435.12 MiB/s 444.97 MiB/s 454.66 MiB/s]
read_full/4.00 MiB      time:   [1.6984 ms 1.7526 ms 1.8037 ms]
                        thrpt:  [2.1657 GiB/s 2.2288 GiB/s 2.3000 GiB/s]
read_full/16.0 MiB      time:   [5.0341 ms 5.2018 ms 5.3719 ms]
                        thrpt:  [2.9087 GiB/s 3.0038 GiB/s 3.1038 GiB/s]

[Xuanwo]

Problem: Stream works well on object storage, but adds extra cost for local fs.