[WIP] Daemon Mode to decrease startup time

[nhoad]

Hi folks! I'm not submitting this as a PR as it's currently in a bit of an unfriendly state test-wise, and I don't want to lock up a bunch of test runners for no good reason.

https://github.com/xonsh/xonsh/compare/master...nathan-hoad:xonshd?expand=1

My branch introduces a way of running a partially initialised xonsh instance as a daemon that clients can connect to and fork off of to complete initialisation - see the commit messages for a description of what's going on. It reduces up start up time quite a bit in my testing, bringing the startup from an average of ~360ms to approximately ~140ms. Measured by putting this in my xonshrc:

g = ${...}.get('_XONSH_START_TIME')
if g:
    import time
    print('xonsh startup', time.time() - float(g))

Starting xonsh regularly:

$ $_XONSH_START_TIME = time.time(); xonsh
xonsh startup 0.3567335605621338

Versus starting it with the daemon:

$ $_XONSH_START_TIME = time.time(); python -m xonsh.xonshd
xonsh startup 0.13603663444519043

Known bugs:

• history doesn't work (it needs to be reinitialised and reloaded, and it's not written out on shutdown)
• doesn't work on windows (but could, I think, using socket.socket.fromshare and socket.socket.share)
• doesn't support anything other than interactive and script_from_stdin modes, and even then the stdin mode is basically interactive, best I can tell.

And surely there are more as well. I'm opening up discussion on this as I think reducing startup time by 200ms, even if a little hacky, is worth talking about :) Aside from the above issues, I've been using this code for a few weeks now with great success.

There's a bunch of print statements left in the code to aid in measuring how long some things take, which are what's messing with the tests - once the branch becomes more stable, I'll rebase the print statements away, of course.

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

[AstraLuma]

Does the --timings flag still work?

[nhoad]

It didn't! Most flags don't work, I'm afraid - quite a lot of them need to be reimplemented. I've pushed another commit that "fixes" --timings though, so now when the client is started with --timings, they'll be displayed:

|----------------|-----------|-----------|                   
|Event name      | Time (s)  | Delta (s) |                   
|----------------|-----------|-----------|                   
|on_ptk_create   |   0.000   |   0.000   |                   
|on_pre_rc       |   0.002   |   0.002   |                   
|on_post_rc      |   0.039   |   0.037   |                   
|on_post_init    |   0.040   |   0.000   |                   
|on_pre_cmdloop  |   0.040   |   0.000   |                   
|on_pre_prompt   |   0.040   |   0.000   |                   
|start           |   0.078   |   0.039   |                   
|----------------|-----------|-----------|

[anki-code]

Great try! But I'm going to close this with reference to #3678.