add lazy loading of dependencies for CLI speedup and remove heavy src/init

[SearchSavior]

@ramonvanraaij, @WizardlyBump17

see if this is faster for you. I couldn't figure out your timing benchmark, but it didnt matter!

running

uv run openarc_cli.py --help

is now nearly instant, but calling through the .toml is much slower.

---

After doing the refactor with Sonnet to lazily import dependencies in the CLI, I continued investigating and discovered that I had this init loaded up with all the core inference classes, the lowest level code with the most imports. Everytime we ran openarc [OPTION] uv would resolve every dependency in each script, which explains why the refactor before 2.0 showed improvment, and why 1.0.5 had no slowdown, because there was no src/init. Anyway, refactor was required, but wasn't the problem.

Whew.

If this works for you guys we can close #37

[WizardlyBump17]

(openarc) root@davi:/OpenArc# time openarc --help
 _____                   ___           
|  _  |                 / _ \          
| | | |_ __   ___ _ __ / /_\ \_ __ ___ 
| | | | '_ \ / _ \ '_ \|  _  | '__/ __|
\ \_/ / |_) |  __/ | | | | | | | | (__ 
 \___/| .__/ \___|_| |_\_| |_/_|  \___|
      | |                              
      |_|                              
 
 Making AI go brr since 2025   

                                                                                                                              
 Usage: openarc [OPTIONS] COMMAND [ARGS]...                                                                                   
                                                                                                                              
 Use this application to interface with the OpenArc server.                                                                   
 Features:                                                                                                                    
 • Start the OpenArc server.                                                                                                  
 • Load models into the OpenArc server.                                                                                       
 • List models from saved configurations.                                                                                     
 • Check the status of loaded models.                                                                                         
 • Unload models.                                                                                                             
 • Benchmark model performance.                                                                                               
 • Query device properties.                                                                                                   
 • Query installed devices.                                                                                                   
 To get started add --help to one of the commands below to view its documentation.                                            
                                                                                                                              
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                            │
│  --help         Show this message and exit.                                                                                │
│                                                                                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ add            - Add a model configuration to the config file.                                                             │
│ bench          - Benchmark inference with pseudo-random input tokens.                                                      │
│ list           - List saved model configurations.                                                                          │
│ load           - Load one or more models from saved configuration.                                                         │
│ serve          - Start the OpenArc server.                                                                                 │
│ status         - GET Status of loaded models.                                                                              │
│ tool           - Utility scripts.                                                                                          │
│ unload         - Unload one or more models from registry and memory.                                                       │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯


real	0m0.204s
user	0m0.146s
sys	0m0.051s

Thank you so much

[SearchSavior]

@WizardlyBump17 How did you get the timing?

[WizardlyBump17]

@WizardlyBump17 How did you get the timing?

time from linux

[SearchSavior]

@WizardlyBump17 How did you get the timing?

time from linux

WizardlyBump of wisdom today, hot damn.

No problem :)

[ramonvanraaij]

Perfect!

 (main) [2]> time openarc --help
 _____                   ___           
|  _  |                 / _ \          
| | | |_ __   ___ _ __ / /_\ \_ __ ___ 
| | | | '_ \ / _ \ '_ \|  _  | '__/ __|
\ \_/ / |_) |  __/ | | | | | | | | (__ 
 \___/| .__/ \___|_| |_\_| |_/_|  \___|
      | |                              
      |_|                              
 
 Making AI go brr since 2025   

                                                                                                                                                                                                                                                                                                        
 Usage: openarc [OPTIONS] COMMAND [ARGS]...                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                        
 Use this application to interface with the OpenArc server.                                                                                                                                                                                                                                             
 Features:                                                                                                                                                                                                                                                                                              
 • Start the OpenArc server.                                                                                                                                                                                                                                                                            
 • Load models into the OpenArc server.                                                                                                                                                                                                                                                                 
 • List models from saved configurations.                                                                                                                                                                                                                                                               
 • Check the status of loaded models.                                                                                                                                                                                                                                                                   
 • Unload models.                                                                                                                                                                                                                                                                                       
 • Benchmark model performance.                                                                                                                                                                                                                                                                         
 • Query device properties.                                                                                                                                                                                                                                                                             
 • Query installed devices.                                                                                                                                                                                                                                                                             
 To get started add --help to one of the commands below to view its documentation.                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                        
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                                                                                                                                                                                      │
│  --help         Show this message and exit.                                                                                                                                                                                                                                                          │
│                                                                                                                                                                                                                                                                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ add                               - Add a model configuration to the config file.                                                                                                                                                                                                                    │
│ bench                             - Benchmark inference with pseudo-random input tokens.                                                                                                                                                                                                             │
│ list                              - List saved model configurations.                                                                                                                                                                                                                                 │
│ load                              - Load one or more models from saved configuration.                                                                                                                                                                                                                │
│ serve                             - Start the OpenArc server.                                                                                                                                                                                                                                        │
│ status                            - GET Status of loaded models.                                                                                                                                                                                                                                     │
│ tool                              - Utility scripts.                                                                                                                                                                                                                                                 │
│ unload                            - Unload one or more models from registry and memory.                                                                                                                                                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯


________________________________________________________
Executed in  129.16 millis    fish           external
   usr time  112.42 millis  416.00 micros  112.00 millis
   sys time   16.06 millis  204.00 micros   15.86 millis

(main)> time openarc help
                                                                                                                                                                                                                                                                                                        
 Usage: openarc [OPTIONS] COMMAND [ARGS]...                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                        
 Try 'openarc --help' for help                                                                                                                                                                                                                                                                          
╭─ Error ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ No such command 'help'.                                                                                                                                                                                                                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                                                                                                                                                                                                        

________________________________________________________
Executed in  128.99 millis    fish           external
   usr time  116.54 millis  599.00 micros  115.94 millis
   sys time   12.00 millis    0.00 micros   12.00 millis