'Content-Length' value returned by TSServer is one off on Windows

[micbou]

The 'Content-Length' value returned by TSServer does not take into account that newlines on Windows are two characters \r\n instead of \n. The issue was found in PR ycm-core/ycmd#156.

Here's a minimal example. Create a dummy test.ts file and a Python script with the following lines:

#!/usr/bin/env python

import json
import subprocess


class TypeScript():
    _tsserver_handle = subprocess.Popen(
      'tsserver.cmd',
      stdout = subprocess.PIPE,
      stdin = subprocess.PIPE
    )

    def SendRequest(self, command, arguments=None):
        request = {
            'seq': 0,
            'type': 'request',
            'command': command
        }
        if arguments:
            request['arguments'] = arguments

        self._tsserver_handle.stdin.write(json.dumps(request) + '\n')


    def ReadResponse(self):
        headers = {}
        while True:
            headerline = self._tsserver_handle.stdout.readline().strip()
            if not headerline:
                break
            key, value = headerline.split( ':', 1 )
            headers[ key.strip() ] = value.strip()

        if 'Content-Length' not in headers:
          raise RuntimeError( "Missing 'Content-Length' header" )
        contentlength = int( headers[ 'Content-Length' ] )
        output = self._tsserver_handle.stdout.read( contentlength )
        print( repr( output ) )


def Main():
    typescript = TypeScript()
    test_file = 'test.ts'

    command = 'open'
    arguments = { 'file': test_file }

    typescript.SendRequest(command, arguments)

    command = 'reload'
    arguments = { 'tmpfile': test_file, 'file': test_file }

    typescript.SendRequest(command, arguments)
    typescript.ReadResponse()

    command = 'reload'
    arguments = { 'tmpfile': test_file, 'file': test_file }

    typescript.SendRequest(command, arguments)
    typescript.ReadResponse()


if __name__ == '__main__':
    Main()

This script sends requests to TSServer and tries to read them. An exception is raised if no Content-Length key in header is found.

On Linux, the output is:

'{"seq":0,"type":"response","command":"reload","request_seq":0,"success":true}\n'
'{"seq":0,"type":"response","command":"reload","request_seq":0,"success":true}\n'

On Windows:

'{"seq":0,"type":"response","command":"reload","request_seq":0,"success":true}\r'
Traceback (most recent call last):
  File "./typescript.py", line 68, in <module>
    Main()
  File "./typescript.py", line 64, in Main
    typescript.ReadResponse()
  File "./typescript.py", line 36, in ReadResponse
    raise RuntimeError( "Missing 'Content-Length' header" )
RuntimeError: Missing 'Content-Length' header

If contentlength is incremented by one:

'{"seq":0,"type":"response","command":"reload","request_seq":0,"success":true}\r\n'
'{"seq":0,"type":"response","command":"reload","request_seq":0,"success":true}\r\n'

In all these cases, TSServer returns a length of 78 characters ({"seq":0,"type":"response","command":"reload","request_seq":0,"success":true} is 77 long). It should return a length of 79 characters on Windows.

[mhegazy]

Content-Length is initialized with 1 all the time, instead it should be host.newLine.length.

 this.sendLineToClient('Content-Length: ' + (1 + Buffer.byteLength(json, 'utf8')) +
                '\r\n\r\n' + json);

[benliddicott]

Alternatively sendLineToClient should either \n or \r\n on both platforms. Since this is an HTTP-like protocol I suggest CRLF for consistency.

In any case I suggest it would be better to send byte-for-byte the same output on all platforms.

(Some background for those not familiar:

HTTP and the web generally favour CRLF (it's specified in the MIME standard and also the format of multiline textboxes in web forms).

OTOH Linux and XML generally favour LF alone.

• It's specified for XML which doesn't even have an option to preserve CR, even though this means there is effectively no such thing as MIME type text/xml (CR is required in text/* and effectively forbidden in XML).
• And in Linux it's effectively forbidden as well (e.g. bash and many other tools will choke on CR as it isn't recognised as harmless whitespace by system() so if it appears on the end of a script line it will be tacked on to the last argument, generally resulting in some sort of failure).

)