Fix SSL_pending() and SSL_has_pending() with DTLS (1.1.1)

[mattcaswell]

This is a backport of #18868 to the 1.1.1 branch.

If app data is received before a Finished message in DTLS then we buffer
it to return later. The function SSL_pending() is supposed to tell you
how much processed app data we have already buffered, and SSL_has_pending()
is supposed to tell you if we have any data buffered (whether processed or
not, and whether app data or not).

Neither SSL_pending() or SSL_has_pending() were taking account of this
DTLS specific app data buffer.

[openssl-machine]

This pull request is ready to merge

[hlandau]

Merged to 1.1.1. Thank you.

[bernd-edlinger]

From the commit message it seems that this change is supposed to only have an effect on DTLS,
but this loop is might have a different return value for TLS with RECORD_LAYER_get_numrpipes > 1

     for (i = 0; i < RECORD_LAYER_get_numrpipes(&s->rlayer); i++) {
         if (SSL3_RECORD_get_type(&s->rlayer.rrec[i])
             != SSL3_RT_APPLICATION_DATA)
-            return 0;
+            return num;
         num += SSL3_RECORD_get_length(&s->rlayer.rrec[i]);
     }

I mean if rrec[0] is APPLICATION_DATA and rrec[1] is not,
then num will be returned while previously 0 was returned.
Is that somehow impossible, or was this intentional?

[mattcaswell]

Pipelined data must always be of the same type, so if the first record is APPLICATION_DATA then the second must be too.

[bernd-edlinger]

Ah, okay, then something like an ossl_assert(i == 0); would make the code more clear?

[mattcaswell]

Yes