Spans may be lost under heavy load

[zhongfox]

this code is from https://github.com/uber/jaeger-client-node/blob/master/src/reporters/udp_sender.js
when high concurrency, spans may be dropped frequently:

    append(span: any): SenderResponse {
        let spanSize: number = this._calcSpanSize(span); 
        if (spanSize > this._maxSpanBytes) {
            return { err: true, numSpans: 1 };
        }

        this._byteBufferSize += spanSize;          // comment 1
        if (this._byteBufferSize <= this._maxSpanBytes) {
            this._batch.spans.push(span);
            if (this._byteBufferSize < this._maxSpanBytes) {
                return {err: false, numSpans: 0};
            }
            return this.flush();
        }

        let flushResponse: SenderResponse = this.flush();  // comment 2
        this._batch.spans.push(span);                                  //comment 3
        this._byteBufferSize = spanSize;
        return flushResponse;
    }

here is a example:

this._maxSpanBytes: 64696
a span lenth: 639
current _byteBufferSize: 64671
now run append(span)

at comment 1: this._byteBufferSize will be 64671+639=65310, it's bigger than this._maxSpanBytes
so it will run comment 2: this.flush()

but in this.flush():

flush(): SenderResponse {
        let numSpans: number = this._batch.spans.length;
        if (numSpans == 0) {
            return {err: false, numSpans: 0}
        }

        let bufferLen = this._byteBufferSize + this._emitSpanBatchOverhead;  //comment 4
        let thriftBuffer = new Buffer(bufferLen);   

comment 4 use this._byteBufferSize, which now is 65310, which will cause "EMSGSIZE" error, if you add a send callback:

this._client.send(thriftBuffer, 0, thriftBuffer.length, this._port, this._host, function(e, sent) {
  console.log(e)
});

and the 64671 byte span is dropped!

fix :

add this._byteBufferSize -= spanSize; before comment 2.
or maybe reconstruct it

this._byteBufferSize += spanSize; and this._batch.spans.push(span); should be atomic, but the code separated them.

[yurishkuro]

yes, it's a bug, thanks for raising it.