Server crash on Windows

[joreiff]

On Windows, the server fails to start up resulting in the following error:

Traceback (most recent call last):
  File "...\Sublime Text 3\Packages\JSONComma\server.py", line 70, in start
    cls.infos = json.loads(line)
  File "./python3.3/json/__init__.py", line 316, in loads
  File "./python3.3/json/decoder.py", line 354, in decode
ValueError: Extra data: line 1 column 5 - line 2 column 1 (char 4 - 66)

This is caused by the server returning

2020/01/13 20:07:06 can't handle arguments when piping from stdin

instead of the expected JSON message.

[math2001]

Oh, that's a mistake from the server (it should always print JSON when the server command is run, because it's going to be parsed as such). I need to set up some tests for that. That errors because I'm doing some detection to see if there is some stuff on stdin, which works on linux but im guessing doesnt on windows... But that also means I'll need to implement something to update the binary from sublime-jsoncomma. That's going to take a bit longer to fix (I should get it done by today).

[math2001]

Damn it, it's not because of windows but because of python 3.3. It works fine on windows with python 3.7, but not on sublime's python...

[math2001]

Sorry for the delay, but this issue raises a fair few other problems. I'm working on what I think will be last one (#12). It's required because this issue requires an (automatic) update of the binary, which I have implemented. But to update the binary, we need to make sure that no process is running, which is a pain to do in Sublime Text because we don't have an on_exit event. The solution I thought of in #12 doesn't even work, it needs to be even more restrictive. And that requires some more changes in how the server works.

So, the fix is coming, it's just taking a bit of time 🙂

[joreiff]

No worries, a proper fix that takes a few days is better than a rushed one. Thanks again for your effort!

[math2001]

It should be fixed now. I'll close this, but could you let me know if you have any issues? Thanks 🙂

[math2001]

It might still be buggy if you still have jsoncomma processes running (you probably do, that's a pain, see #12 ). The updates won't work, and there isn't much this plugin can do about that. The only way that is somewhat sane is for you to:

• manually kill every jsoncomma process (on windows, open up cmd and run taskkill /IM jsoncomma.exe, on linux pkill jsoncomma, on mac I'm not sure)
• or reboot your computer

once you've done either one of these, make sure you have the latest version of this plugin installed, and start ST. Everything should now work just fine 🙂

[math2001]

I've just tested on windows, and it works, but it's blocking the editor on save (like for a sec). Doing a simple post requests in the console:

import requests
resp = requests.post("http://localhost:2442", data="""{"some": "json"}""")

blocks for a second. It's horrible. Could you tell me if you experience the same thing? @joreiff

[joreiff]

I do not have access to my Windows machine right now, but I will get back to you this evening.

[joreiff]

I can confirm that the plugin now runs correctly under Windows. And I also get an approximately one-second pause on save. I played around a bit and it seems that the issue is Python itself. Both requests and urllib.request show the same delay. Using the PowerShell, however, I can execute a request almost instantly:

Invoke-WebRequest -Method Post -Uri http://localhost:2442 -Body '{"some": "json"}'

Interestingly enough, this only affects Python clients, a Python server started via

python -m http.server 8000

responds much quicker.

Have you tried reusing the HTTP connection to the server (keep-alive)? Some quick experimentation suggests that this could mitigate the problem somewhat (IPython):

with requests.Session() as s:
    for t in (0, 1, 2, 4, 58, 62):
        time.sleep(t)
        print('After {}s sleep'.format(t))
        %time s.post("http://localhost:2442", data="""{"some": "json"}""")

I was, however, not able to keep the connection open for more than about one minute, neither through TCP, nor HTTP Keep-Alive timeout settings. Crazy idea: Periodically send "Keep-Alive" GET requests from a background thread (e.g. using sublime.set_timeout_async) while a JSON document is opened. On the other, that may be total overkill... 😉

[math2001]

I found the issue and the solution!

The problem comes from how windows looks up the server. Windows tries both IPv4 and IPv6, sequentially with a one second timeout. And what is currently happening (ie. before the fix) is that the server is told to listen only on localhost, which I'm guessing Go says is ::1, ie. IPv6 and not IPv4. But then, requests uses IPv4 by default. So, it pings 127.0.0.1, waits for the 1 second timeout, and then tries IPv6 ::1, and it works.

The reason python -m http.server 8000 works quickly is because since you are not specifying a port, it listens on every interface (i'm not 100% confident about this topic, take this with a grain of salt), so on your loopback interface IPv4/IPv6, on your "wifi" interface (ie local wifi address, probably 192.168.x.x), IPv4/IPv6 again, on your ethernet interface, etc... So, that means that whichever IP version the client uses, it responds straight away (without a lag).

So, the solution is either to start the server on every interface (ie jsoncomma server -host "" -port <myport>) so that we run on both IPv4 and IPv6, or to explicitly tell the server AND client to use a certain IP version.

I picked the second one because I don't want the jsoncomma server to be accessible by people on the user's local network (ie. having available on 192.168.x.x), I just want it to run on 127.0.0.1.

The fix is literally a one line change, replace localhost (which allows the OS to pick whatever it prefers) by 127.0.0.1, ie. force IPv4. I would have liked to enforce IPv6, but it seems like it's a pain with requests. I might look it into that later, but honestly, it works now, and if it has a chance of not working because of it, I'm sure requests will make it easy to use IPv6.

I'll push the fix in a sec. Thank you @joreiff for you time and efforts, I really appreciate it ♥️