[Core][Streaming Generator] Fix memory leak from the end of object stream object

[rkooo567]

Why are these changes needed?

ray/src/ray/core_worker/core_worker.h

Line 446 in 4304e45

// TODO(sang): This seems bad... We should delete the memory store

Currently, the objects are deleted from the memory store manually when the reference is removed (via __dealloc__ from ObjectRef class here;

ray/python/ray/includes/object_ref.pxi

Line 62 in 4304e45

worker.core_worker.remove_object_ref_reference(self)

).

This causes the leak from streaming generator because it doesn't actually create a Cython object ref class for the end of stream.

This PR fixes the issue by manually deleting the objects from memory store when the ref of streaming generator objects goes to 0.

In the long term, we should fix this by modifying reference_count.cc to actually delete the object from the memory store. I will follow up with this, but I chose the current solution as a short term because we should merge this by 2.6.3 (which we try to branch cut by Sun)

Related issue number

Closes #38089

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[edoakes]

Feels like there are cpp unit tests missing here to verify that the ref counting works as expected

[edoakes]

@rkooo567 did you re-run the Serve repro to verify it's fixed? I'll kick off a long run on workspaces using these wheels too just to be sure.

[edoakes]

Looks good

[rkooo567]

Feels like there are cpp unit tests missing here to verify that the ref counting works as expected

Good catch... Adding it now.

Btw, is it possible to add that long-running test as a regular long running test or some sort of release tests?

[rkooo567]

Unit tests are added