refactor(amalgamate): remove amalgamation

[gforsyth]

I think the time for this has probably come.
I am interested if folks can try out this branch using xonsh --no-rc --timings and compare it to main and see if there's a noticeable performance difference.

I suspect most of the startup time penalty will be a wash, with more benefits coming from the lazy evaluation of our compiled regexes, but I want to make sure we aren't completely borking startup time.

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

[jnoortheen]

There is not much difference on my machine (Python 3.10 + M1 Air),

On this branch - 0.160s

Event name	Time (s)	Delta (s)
start	0.000	0.000
pre_execer_init	0.004	0.004
post_execer_init	0.031	0.027
pre_xontribs_autoload	0.031	0.000
post_xontribs_autoload	0.055	0.024
on_pre_rc	0.055	0.000
on_post_rc	0.055	0.000
on_ptk_create	0.113	0.058
on_post_init	0.113	0.000
on_pre_cmdloop	0.113	0.000
on_pre_prompt_format	0.113	0.000
on_pre_prompt_tokenize	0.133	0.020
on_post_prompt_tokenize	0.140	0.007
on_pre_prompt_style	0.159	0.019
on_post_prompt_style	0.160	0.001
on_pre_prompt	0.160	0.000

On main - 0.166s

Event name	Time (s)	Delta (s)
start	0.000	0.000
pre_execer_init	0.007	0.007
post_execer_init	0.034	0.027
pre_xontribs_autoload	0.034	0.000
post_xontribs_autoload	0.058	0.024
on_pre_rc	0.058	0.000
on_post_rc	0.058	0.000
on_ptk_create	0.124	0.066
on_post_init	0.124	0.000
on_pre_cmdloop	0.124	0.000
on_pre_prompt_format	0.124	0.000
on_pre_prompt_tokenize	0.141	0.017
on_post_prompt_tokenize	0.147	0.005
on_pre_prompt_style	0.165	0.018
on_post_prompt_style	0.166	0.001
on_pre_prompt	0.166	0.000

[jnoortheen]

Lets see what is the difference for others

[Techcable]

2017 ArchLinux Lenovo Laptop (4 cores, 8GB RAM)

On this branch - 0.287s

0.556s for first run

 Debug level: Off
|-------------------------|-----------|-----------|
|Event name               | Time (s)  | Delta (s) |
|-------------------------|-----------|-----------|
|start                    |   0.000   |   0.000   |
|pre_execer_init          |   0.008   |   0.008   |
|post_execer_init         |   0.084   |   0.077   |
|pre_xontribs_autoload    |   0.085   |   0.000   |
|post_xontribs_autoload   |   0.112   |   0.027   |
|on_pre_rc                |   0.112   |   0.000   |
|on_post_rc               |   0.112   |   0.000   |
|on_ptk_create            |   0.208   |   0.096   |
|on_post_init             |   0.208   |   0.000   |
|on_pre_cmdloop           |   0.208   |   0.000   |
|on_pre_prompt_format     |   0.208   |   0.000   |
|on_pre_prompt_tokenize   |   0.237   |   0.029   |
|on_post_prompt_tokenize  |   0.246   |   0.009   |
|on_pre_prompt_style      |   0.285   |   0.039   |
|on_post_prompt_style     |   0.287   |   0.002   |
|on_pre_prompt            |   0.287   |   0.000   |
|-------------------------|-----------|-----------|

On main - 0.284s

0.606s for first run

 Debug level: Off
|-------------------------|-----------|-----------|
|Event name               | Time (s)  | Delta (s) |
|-------------------------|-----------|-----------|
|start                    |   0.000   |   0.000   |
|pre_execer_init          |   0.008   |   0.008   |
|post_execer_init         |   0.084   |   0.076   |
|pre_xontribs_autoload    |   0.084   |   0.000   |
|post_xontribs_autoload   |   0.111   |   0.027   |
|on_pre_rc                |   0.111   |   0.000   |
|on_post_rc               |   0.111   |   0.000   |
|on_ptk_create            |   0.206   |   0.095   |
|on_post_init             |   0.206   |   0.000   |
|on_pre_cmdloop           |   0.207   |   0.000   |
|on_pre_prompt_format     |   0.207   |   0.000   |
|on_pre_prompt_tokenize   |   0.237   |   0.031   |
|on_post_prompt_tokenize  |   0.244   |   0.007   |
|on_pre_prompt_style      |   0.282   |   0.038   |
|on_post_prompt_style     |   0.284   |   0.002   |
|on_pre_prompt            |   0.284   |   0.000   |
|-------------------------|-----------|-----------|

On my 2021 M1 Pro Macbook Pro (10 cores, 32 GB RAM)

On this branch - 0.075s

0.273s for first run

 Debug level: Off
|------------------------|-----------|-----------|
|Event name              | Time (s)  | Delta (s) |
|------------------------|-----------|-----------|
|start                   |   0.000   |   0.000   |
|pre_execer_init         |   0.004   |   0.004   |
|post_execer_init        |   0.049   |   0.045   |
|pre_xontribs_autoload   |   0.049   |   0.000   |
|post_xontribs_autoload  |   0.055   |   0.006   |
|on_pre_rc               |   0.055   |   0.000   |
|on_post_rc              |   0.055   |   0.000   |
|on_post_init            |   0.060   |   0.004   |
|on_pre_cmdloop          |   0.060   |   0.000   |
|on_pre_prompt_format    |   0.060   |   0.000   |
|on_pre_prompt           |   0.075   |   0.015   |
|------------------------|-----------|-----------|

On main - 0.077s

0.246 for first run

Debug level: Off
|------------------------|-----------|-----------|
|Event name              | Time (s)  | Delta (s) |
|------------------------|-----------|-----------|
|start                   |   0.000   |   0.000   |
|pre_execer_init         |   0.004   |   0.004   |
|post_execer_init        |   0.049   |   0.044   |
|pre_xontribs_autoload   |   0.049   |   0.000   |
|post_xontribs_autoload  |   0.055   |   0.006   |
|on_pre_rc               |   0.055   |   0.000   |
|on_post_rc              |   0.055   |   0.000   |
|on_post_init            |   0.059   |   0.004   |
|on_pre_cmdloop          |   0.059   |   0.000   |
|on_pre_prompt_format    |   0.059   |   0.000   |
|on_pre_prompt           |   0.077   |   0.018   |
|------------------------|-----------|-----------|

Conclusions

Both tests ran on Python 3.10, both laptops were plugged in and had no other apps running (besides a few browser tabs).

I did one warmup run and took the second result.

My old 2017 laptop has much slower results (as expected), but on both laptops it appears the difference between branches is not statistically significant. One laptop points one way and one points the other (and further runs only give more inconclusive results).

[jnoortheen]

I have resolved the conflicts.

[codecov-commenter]

Codecov Report

Merging #4858 (7de8b88) into main (cbf23e6) will decrease coverage by 1.89%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #4858      +/-   ##
==========================================
- Coverage   66.57%   64.68%   -1.90%     
==========================================
  Files         133      129       -4     
  Lines       23722    23665      -57     
  Branches     4705     4700       -5     
==========================================
- Hits        15794    15308     -486     
- Misses       6694     7115     +421     
- Partials     1234     1242       +8     

Flag	Coverage Δ
macOS-latest	63.66% <0.00%> (+0.09%)	⬆️
ubuntu-latest	64.29% <0.00%> (+0.09%)	⬆️
windows-latest	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
xonsh/__init__.py	100.00% <ø> (+66.66%)	⬆️
xonsh/environ.py	81.34% <ø> (-1.11%)	⬇️
xonsh/winutils.py	0.00% <0.00%> (-47.81%)	⬇️
xonsh/prompt/job.py	20.00% <0.00%> (-50.00%)	⬇️
xonsh/platform.py	48.68% <0.00%> (-15.55%)	⬇️
xonsh/completers/path.py	56.41% <0.00%> (-15.39%)	⬇️
xonsh/procs/readers.py	47.00% <0.00%> (-11.06%)	⬇️
xonsh/xoreutils/_which.py	27.56% <0.00%> (-10.26%)	⬇️
xonsh/lazyimps.py	62.29% <0.00%> (-9.84%)	⬇️
... and 27 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[jnoortheen]

@anki-code could you test this and report ?

[anki-code]

No sufficient difference on my PC.

[jnoortheen]

well 3 is the magik number!