Analyzer - Generate baseline given results from multiple nodes.

[guoshzhao]

Description
Generate baseline given results from multiple nodes.

Major Revision

• Add sub command sb result generate-baseline
• Add UT and docs

[codecov]

Codecov Report

Attention: 43 lines in your changes are missing coverage. Please review.

Comparison is base (f53d941) 86.96% compared to head (fea58d5) 86.56%.

Files	Patch %	Lines
superbench/analyzer/baseline_generation.py	73.63%	29 Missing ⚠️
superbench/cli/_result_handler.py	13.33%	13 Missing ⚠️
superbench/analyzer/file_handler.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #575      +/-   ##
==========================================
- Coverage   86.96%   86.56%   -0.40%     
==========================================
  Files          93       94       +1     
  Lines        6266     6396     +130     
==========================================
+ Hits         5449     5537      +88     
- Misses        817      859      +42     

Flag	Coverage Δ
cpu-python3.6-unit-test	71.45% <52.27%> (-0.41%)	⬇️
cpu-python3.7-unit-test	71.45% <52.27%> (-0.41%)	⬇️
cpu-python3.8-unit-test	71.92% <52.27%> (-0.41%)	⬇️
cuda-unit-test	84.56% <67.42%> (-0.35%)	⬇️
directx-unit-test	35.19% <15.90%> (-0.42%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yukirora]

@guoshzhao , could u please solve the UT failure ERROR: Could not find a version that satisfies the requirement numpy>=1.20.3 (from superbench[cpuworker,test]) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5) ERROR: No matching distribution found for numpy>=1.20.3 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available. You should consider upgrading via the '/usr/local/bin/python3 -m pip install --upgrade pip' command.