Clean example artifacts and switch test outputs to result filenames#668
Clean example artifacts and switch test outputs to result filenames#668ssss141414 wants to merge 62 commits into
Conversation
DingmaomaoBJTU
force-pushed
the
shzhen/examples_20_mainclean
branch
from
7b12b8a to
5b7ddc4
Compare
Source build configs for CLIP-style dual-encoder models are split into per-encoder files (build_config_image-encoder.json, build_config_text-encoder.json) rather than a single build_config.json. The previous import script defaulted to fp16 when no top-level build_config.json was found, mislabeling this quantized run. Verified actual precision is w8a16 (uint8 weights, uint16 activations) from the per-encoder configs.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| import argparse | ||
| import json | ||
| import re |
|
Repo size concern — please hold before merging This PR adds 2,790 new blobs / +6.07 MB uncompressed to the pack. No individual file crosses 100 KB (max ~78 KB), but the aggregate exceeds the entire current Most of it is per-(EP × model × task) generated artifacts: Could we keep these out of the source repo — generate them in CI from the existing scripts and publish as a pipeline artifact / release asset, and add the matching patterns to |
…rosoft/winml-cli into shzhen/examples_20_mainclean
…ports and summary
Adds configs for 29 (model, task) tuples to dml/gpu, mlas/cpu, nv_tensorrt_rtx/gpu, openvino/{cpu,gpu,npu}, qnn/{gpu,npu}, vitisai/npu. Updates scripts/generate_example_configs.py MODELS (63 -> 92). Regenerates all REPORT.md and examples/summary.md from real config/result files.
…ross 9 EPs Pulled latency_ms (mean/min/max/p50/p90/p95/p99/std), throughput.samples_per_sec, model_info (input/output names/shapes/types), and iterations/warmup from the e2e_model_coverage_result bundle. raw_samples_ms and warmup_mean stay empty/null (not in source). Reports and summary regenerated from real files.
…ny-precision eval-pass)
…eval pass (any precision)
…maries - Restore 132 previously-deleted VitisAI NPU fp16 artifacts (configs, eval results, errors, timeouts) - Harvest 30 fp16 perf results from external bundle - Regenerate REPORT.md, summary.md, summary_2.md, eval_summary.md - Drop Builtin Models section from examples/summary.md and rebuild script
…rosoft/winml-cli into shzhen/examples_20_mainclean
…rosoft/winml-cli into shzhen/examples_20_mainclean
…rosoft/winml-cli into shzhen/examples_20_mainclean
…ports/summaries, fix eval_summary link paths
…EP configs now byte-identical
DingmaomaoBJTU
left a comment
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Review Notes
This PR adds 300+ example artifact files (~291K lines) with no description. A few concerns:
-
Missing PR description - please add context on why these artifacts are being checked in (golden test references? documentation? CI fixtures?).
-
Repo bloat - 291K lines of JSON artifacts committed to the repo will inflate clone size permanently. Consider using Git LFS or storing in a CI artifact bucket.
-
Error files checked in - several
*_eval_result.error.txtfiles contain stack traces. Are these intentional golden files for expected failures, or accidental inclusions? -
Title mismatch - title says "switch test outputs to result filenames" but the diff is purely additions with 0 deletions. Was there a force-push that squashed the rename/delete step?
Please address these concerns before this can be approved.
5 participants
No description provided.