perf(sdk): improve file descriptor management

[dmitryduev]

Fixes WB-11427 (probably not fully, but this should be a good starting point)
Fixes #2825

Description

Reduce the default number of open file descriptors:

• default to lazily loading optional external modules:
  • make wandb.util.get_module lazily load modules by default
  • wandb.old.summary was loading h5py, which many users will never need. It was accounting for ~40% of open file descriptors when you simply say import wandb.

With the above two improvements, for a simple script junk.py like this:

import time

import wandb

run = wandb.init()
time.sleep(5)
run.finish()

p=`ps -ef | grep "junk" | awk 'NR==1 {print $2}'`
lsof -p $p | wc -l 

goes down from ~115 to ~70.

• Make sure to close the internal log file when finishing the internal process

Currently, for a script like this:

import wandb

for i in range(20):
    print(i)
    with wandb.init():
        pass

All the debug-internal.log files stay open until the service process exits -- fixed this.

Testing

How was this PR tested?

Checklist

• Include reference to internal ticket "Fixes WB-NNNN" and/or GitHub issue "Fixes #NNNN" (if applicable)
• Ensure PR title compliance with the conventional commits standards

[codecov]

Codecov Report

Merging #4617 (669ba01) into main (39d50c3) will increase coverage by 0.05%.
The diff coverage is 96.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4617      +/-   ##
==========================================
+ Coverage   83.19%   83.24%   +0.05%     
==========================================
  Files         279      279              
  Lines       34903    34925      +22     
==========================================
+ Hits        29038    29075      +37     
+ Misses       5865     5850      -15     

Flag	Coverage Δ
functest	55.98% <92.00%> (-0.10%)	⬇️
unittest	73.82% <96.00%> (+0.08%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
wandb/__init__.py	91.34% <ø> (ø)
wandb/data_types.py	82.86% <ø> (ø)
wandb/util.py	87.80% <94.44%> (+0.50%)	⬆️
wandb/apis/public.py	81.87% <100.00%> (ø)
wandb/sdk/internal/internal.py	87.22% <100.00%> (+0.44%)	⬆️
wandb/sdk/lib/git.py	77.71% <0.00%> (ø)
wandb/integration/tensorboard/monkeypatch.py	91.35% <0.00%> (ø)
wandb/sdk/wandb_run.py	90.89% <0.00%> (+0.23%)	⬆️
wandb/sdk/wandb_setup.py	88.94% <0.00%> (+0.50%)	⬆️
... and 2 more

[dmitryduev]

should this be True by default?