[Datasets] Remove stats actor

[c21]

What happened + What you expected to happen

Stats actor is responsbile to keep the stats for read tasks only, and be accessed by driver when driver RPC to stats actor. This model has the weird timing issue that if ds.stats() is called immediately after read, the stats may not be up-dated. The reason is when ds.stats() is called and stats actor does not get the metadata from read tasks yet. Read tasks makes RPC to stats actor for sending the metadata. The read stats is stored in stats actor, and fetched into driver memory. so causes this timing issue here.

Plan here is to kill stats actor and move the read stats into driver memory directly.

Versions / Dependencies

master

Reproduction script

on master, this script may or may not print execution stats depending on timing:

import ray
import time

ds = ray.data.range_table(5)

for batch in ds.iter_batches():
    print(batch)

#time.sleep(3)

print(ds.stats())
print(ds.stats())

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[jianoaix]

Just to note that to keep parity after removal, we need to improve the walltime tracking for stages in block metadata, potentially recording event timestamps. @scottjlee

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[scottjlee]

keep

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.