Readable event triggers but socket.readableLength returns zero

[davidkjackson54]

Version

16.4.2

Platform

Windows 10

Subsystem

No response

What steps will reproduce the bug?

I am reading specific lengths of data off the readable queue. I check the readableLength to determine how much data I can read.
However, there are occasions when the readable event triggers but the readableLength is returned as zero.

client._socket.on("readable", () => {
        console.log("Readable event triggered");
        console.log(
          `${new Date().toLocaleString()}, Readable Stream has ${client._socket.readableLength} bytes of data available`
        );
        if (client._socket.readableLength === 0) {
          // for some reason the readable event can trigger but there be nothing in the readable queue
          console.log("readable queue is empty");
          return;
        }

How often does it reproduce? Is there a required condition?

This typically happens when TCP has decided to chunk the return of the data. I read what is available then wait for the next readable event to fire, then I find that the readableLength is zero - despite the readable event having triggered.

What is the expected behavior?

readableLength should contain the number of bytes available in the stream after the readable event has triggered.

What do you see instead?

Readable event triggered
9/17/2021, 7:47:19 AM, Readable Stream has 0 bytes of data available
readable queue is empty

Additional information

No response

[ronag]

I believe 'readable' can also be emitted when the stream has "ended", i.e. readableEnded is true.

[ronag]

@nodejs/streams I think we have an edge case where an empty buffer can cause readable to be emitted... we should probably ignore empty chunks?

[ronag]

I don't think this is a bug though...

[davidkjackson54]

I have the "end" event in place but I am not seeing it being triggered.
How can the readable event be triggered when there is nothing to actually read?? That seems like a bug to me..
Otherwise it would appear to defeat the purpose of the event.

Thee doc states:
Event: 'readable'#
History
The 'readable' event is emitted when there is data available to be read from the stream. In some cases, attaching a listener for the 'readable' event will cause some amount of data to be read into an internal buffer.

[mcollina]

Consider this example:

const { Readable } = require('stream')

const r = new Readable({
  read () {}
})

r.on('readable', function () {
  console.log('readable', r.read())
})

r.on('end', function () {
  console.log('end')
})

r.push(null)

null implies EOF and it will trigger 'readable' and 'end'.
There is something to read on the stream, the end of it.

I think we could improve the docs.

[lpinca]

I have the "end" event in place but I am not seeing it being triggered.

@davidkjackson54 are you are to write a reproducible example using only Node.js core modules?

[davidkjackson54]

Unfortunately, it would be quite difficult to create a recreation scenario.
However, as I say, I do have the 'end' event in place and it is not being triggered.
Also, as I mentioned above, despite the readable event being triggered, I am now having to check the length of socket.readableLength and manage it.

I honestly think that it should not trigger the readable event when there is no actual data to read as signified by a readableLength of zero being returned.

[axaysushir]

Should I take this issue? Also, can anyone suggest to me how to get started with this?

[jasnell]

@ronag or @mcollina may have some ideas.

[mcollina]

In https://github.com/nodejs/node/blob/master/doc/api/stream.md#event-readable, we mention

The 'readable' event will also be emitted once the end of the stream data has been reached but before the 'end' event is emitted.

We should note that this implies that .read() will return null in that case.

[axaysushir]

Ok, I'll look at it

[davidkjackson54]

As I say, the end event is not being fired... when the zero length readable event has fired.

[mcollina]

@davidkjackson54 unfortunately we cannot help unless you can produce a script to reproduce the problem you are facing.

Calling .read() will cause end to be emitted.

[davidkjackson54]

It is not possible due to the data that I am retrieving,. This is coming from an IBM mainframe via TCP sockets and I cannot carve out the code to a recreated scenario.
As I say , the 'readable' event is being triggered and the socket.readableLength from that is coming back as zero. No 'end' event is being triggered for that particular situation.

[davidkjackson54]

Just to be clear , I am getting the readable event trigger, I check the length - it is zero so I don’t bother calling the read. So probably why the end event is not being triggered. Per the doc- readable infers something to read. Nothing to read if the readable length is zero is the way I interpret that….. David

…

On Fri, Sep 24, 2021 at 2:24 PM Matteo Collina ***@***.***> wrote: @davidkjackson54 <https://github.com/davidkjackson54> unfortunately we cannot help unless you can produce a script to reproduce the problem you are facing. Calling .read() will cause end to be emitted. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALAIHNBNN5F4WX2O2BYIUV3UDTUCHANCNFSM5EHQWNTQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[vweevers]

Per the doc- readable infers something to read. Nothing to read if the readable length is zero is the way I interpret that

Readable infers new information. This is explained in the documentation:

The 'readable' event is emitted when there is data available to be read from the stream. [..] The 'readable' event will also be emitted once the end of the stream data has been reached but before the 'end' event is emitted. [..] Effectively, the 'readable' event indicates that the stream has new information: either new data is available or the end of the stream has been reached. In the former case, stream.read() will return the available data. In the latter case, stream.read() will returnnull.

Note @mcollina, this means there's no need for #40136 (comment). Everything stated in this thread is also present in documentation. But it could be improved by reordering the sentences, to give more information up front. For example:

The 'readable' event is emitted when there is data available to be read from the stream or when the end of the stream has been reached. Effectively, the 'readable' event indicates that the stream has new information. If data is available, stream.read() will return that data. [..] If the end of the stream data has been reached, calling stream.read() will return null, which in turn triggers the 'end' event.

[mcollina]

That would be perfect @vweevers.

[davidkjackson54]

Thanks David

…

On Sat, Sep 25, 2021 at 12:18 AM Matteo Collina ***@***.***> wrote: That would be perfect @vweevers <https://github.com/vweevers>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALAIHNGWNXZ6MDCRUGEXRGDUDVZVVANCNFSM5EHQWNTQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[davidkjackson54]

When the readable event triggers, I am only reading specific lengths of data from the stream. I do not retrieve the entire stream in one read. If the readable event triggers I check how much data is available to read in case it is less than what I am requiring and I accumulate the chunks till I have the length I require. If the read event triggers but the readable length is zero and I have to perform the read then it would have to be a read of zero in order to the trigger the end event and then I have to wait for more data to flow and trigger the next readable event. Will a read attempt of zero cause that to happen?

…

On Fri, Sep 24, 2021 at 2:24 PM Matteo Collina ***@***.***> wrote: @davidkjackson54 <https://github.com/davidkjackson54> unfortunately we cannot help unless you can produce a script to reproduce the problem you are facing. Calling .read() will cause end to be emitted. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALAIHNBNN5F4WX2O2BYIUV3UDTUCHANCNFSM5EHQWNTQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[mcollina]

Yes, it does emit 'end' when stream.read(0) is called.

const { Readable } = require('stream')

const r = new Readable({
  read () {}
})

r.on('readable', function () {
  console.log('readable', r.read(0))
})

r.on('end', function () {
  console.log('end')
})

r.push(null)