Issue summarization for example of `MOT17/multiedge_inference_bench`

[suhaan-24]

Author: suhaan-24
Pre-task 1: LFX Mentorship 2026 Term 2
Date: 2026-05-21

Scope: This analysis extends prior work by covering 9 tracked GitHub issues (including 4 newly filed in 2026) and 13 original static-analysis findings not present in any prior submission or automated review. The closed LFX Term 3 2025 pre-task #231 is included as historical context; all findings in that submission that remain unresolved are re-examined here with updated evidence from the current main branch. Gemini-code-assist has reviewed PRs #392, #398, #400, #402, #336, and #309; every section below that overlaps with a gemini comment explicitly labels the overlap and states what this analysis adds beyond it. 13 original static-analysis findings have been identified and filed as GitHub issues (#443, #444, #445, #447, #448, #449, #450, and a comment on #391), none of which were present in any prior submission or automated review.

---

1. Background

The MOT17/multiedge_inference_bench example benchmarks pedestrian tracking and re-identification (ReID) in a cloud-edge collaborative setting using the MOT17 dataset. It exercises ianvs's MultiEdge Inference paradigm by running two sequential benchmark jobs: a tracking job (ByteTrack detector) and a ReID job (M3L feature extractor), then merging the results into a PDF report via generate_reports.py. This example is the sole representative of paradigm_type: multiedgeinference in the entire ianvs example suite, making every bug in it a unique barrier to validating this paradigm.

As of 2026-05-21, the example is entirely non-runnable due to a combination of stale path references, runtime crashes, missing dependencies, and core framework regressions. The section below catalogues every relevant GitHub issue, followed by 13 original static-analysis findings, now filed as GitHub issues (#443–#450 and a comment on #391).

SPECIFIC GREEN CHANNEL

---

2. Table of Contents — All Linked Issues

#	Issue	Category	Status	Link
1	MOT17 example fails to run due to broken "pedestrian_tracking" path references	Path / Configuration	Open	#391
2	MOT17 tracking_job & reid_job fail to launch due to nested directory mismatches	Path / Configuration	Open	#397
3	NameError in MOT17 generate_reports.py caused by undefined config_file variable	Runtime Errors / Crashes	Open	#399
4	ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero	Runtime Errors / Crashes	Open	#401
5	'MultiedgeInference' object has no attribute 'modules_funcs'	Core Framework	Open	#93
6	[Core Framework] Incorrect logical operator in _check_fields validation across examples	Core Framework	Open	#335
7	Inconsistent with interface name initial_model_url and environment variable base_model_url	Naming / API Bug	Open	#14
8	Complete unclear dependencies for Ianvs	Missing Dependencies / Docs	Open	#132
9	PRE-TASK: MOT17 Multi-Edge Inference Benchmark (LFX TERM 3 2025)	Historical / Prior Work	Closed	#231
10	cmc.py line 48: hardcoded stale output path	Path / Config Bug	Open	comment on #391
11	ByteTrack basemodel.py: DDP and dist never imported — NameError in distributed mode	Algorithm Code Bug	Open	#443
12	ByteTrack basemodel.py: tautological assertion and malformed YAML indentation	Algorithm Code Bug / Config Bug	Open	#444
13	M3L basemodel.py: unguarded regex and deprecated addmm_ API removed in PyTorch ≥ 2.0	Algorithm Code Bug	Open	#445
14	ReID metrics: IndexError and RuntimeError on edge case inputs	Runtime Error	Open	#447
15	NaN propagation from 5 upstream tracking metrics into f1_score.py	Runtime Error	Open	#448
16	Missing requirements.txt and Docker-specific workspace path block bare-metal setup	Missing Dependency / Portability Bug	Open	#449
17	multiedge_inference.py: unconditional import onnx fails for non-partitioning users	Core Framework Bug	Open	#450

---

3. Detailed Summarization for Each Issue

---

Issue #391 — MOT17 example fails to run due to broken "pedestrian_tracking" path references

Introduction:
When ianvs was refactored, the top-level example directory was renamed from pedestrian_tracking/ to MOT17/multiedge_inference_bench/pedestrian_tracking/, but no path references inside configuration and script files were updated. Every testenv, algorithm url, and metric url field in the YAML configs still points to ./examples/pedestrian_tracking/multiedge_inference_bench/…, a path that does not exist. The benchmarking job immediately exits with FileNotFoundError before any inference begins.

Prior review overlap:
Gemini flagged (in PR #392): Hardcoded paths in generate_reports.py and cmc.py (both marked as resolved in the PR branch).

New contribution in this analysis:
This analysis extends beyond that by: (a) running grep -r "pedestrian_tracking" examples/MOT17/ on the current main branch to confirm 25 hits across 9 files remain unfixed; (b) identifying that cmc.py line 48 carries a distinct stale path (./examples/pedestrian_tracking/multiedge_inference_bench/cmc) that must be updated at a different depth than the YAML paths; (c) showing that the stale cmc.py path causes a secondary crash in generate_reports.py line 64 when it tries to embed the missing CMC plot in the PDF (self.image(reid_result["cmc"], ...)); (d) noting that gemini's comments were vague — no exact replacement paths were specified.

Challenge:
The rename is pervasive: 25+ references across 9 files. A naive find-and-replace of pedestrian_tracking/ → MOT17/multiedge_inference_bench/pedestrian_tracking/ is insufficient because cmc.py's output directory path embeds the old structure at a different nesting level. PR #392 and PR #398 both address overlapping subsets, creating merge-ordering risk.

Motivation:
Highest-severity blocker: no user can start either benchmark job without this fix. It makes the multiedgeinference paradigm entirely unvalidatable.

Linked PRs: #392 (open), #398 (open)

Additional insights:
Verification command: grep -r "pedestrian_tracking" examples/MOT17/ — should return zero results after all PRs are merged.

---

Issue #397 — tracking_job & reid_job fail to launch due to nested directory mismatches

Introduction:
tracking_job.yaml and reid_job.yaml reference testenv and algorithm paths using the pre-rename structure. Both fail immediately when passed to ianvs -f with FileNotFoundError.

Prior review overlap:
Gemini flagged (in PR #398): Gemini's only comment on PR #398 was about cloud-edge-collaborative-inference-for-llm/testenv/accuracy.py (a zip() suggestion for LLM metrics). Gemini made no comment on the MOT17-specific path changes within PR #398.

New contribution in this analysis:
This analysis extends beyond that by: independently identifying that PR #398 is over-scoped (12 commits, 4 distinct examples), that the MOT17 path fix is buried among unrelated LLM example changes, and that MPS device-detection logic added in the same PR will raise AttributeError on PyTorch < 1.12.

Challenge:
Issues #391 and #397 share the same root cause but were filed by different authors (ARYANPATEL-BIT and rakshaak29). Two partially-overlapping PRs (#392 and #398) were opened in response. A reviewer must verify after both PRs land that zero stale references remain.

Motivation:
tracking_job.yaml and reid_job.yaml are the CLI entry-point files. Breakage here means the example cannot start at all.

Linked PRs: #398 (open)

---

Issue #399 — NameError in MOT17 generate_reports.py

Introduction:
Lines 77 and 86 of generate_reports.py reference the undefined variable config_file instead of the locally-scoped tracking_config_file (line 75) and reid_config_file (line 84). When either config file is missing, Python raises NameError: name 'config_file' is not defined.

Prior review overlap:
Gemini flagged (in PR #400): Even after fixing the NameError, a TypeError will occur if the argument is None, because utils.is_local_file(None) calls os.path.isfile(None), which is invalid. Gemini recommended adding argument validation before the filesystem check.

New contribution in this analysis:
This analysis extends beyond that by: identifying two additional unfixed bugs in the same function that PR #400 does not address: (a) the stale output directory on line 93 (./examples/pedestrian_tracking/multiedge_inference_bench/reports); (b) the deprecated Pandas argument delim_whitespace=True on lines 80 and 88 (deprecated since Pandas 2.0, April 2023). The complete fix for generate_reports.py requires four independent changes, not two.

Challenge:
This is a secondary failure mode: it surfaces only after path fixes (#391, #397) are merged and both benchmark jobs run successfully. Its late position in the workflow means it wastes the user's full GPU run time before failing.

Motivation:
Step 3 of the three-step MOT17 workflow. Failure here leaves the user with no consolidated report despite successful benchmarks.

Linked PRs: #400 (open)

Additional insights:
Complete generate_reports.py fix checklist: (1) rename config_file → tracking_config_file/reid_config_file; (2) add null guard before is_local_file(); (3) fix stale output_dir on line 93; (4) replace delim_whitespace=True with sep=r'\s+'.

---

Issue #401 — ZeroDivisionError in MOT17 f1_score.py

Introduction:
f1_score.py line 51 computes 2 * (precision * recall) / (precision + recall) without any guard. When a model produces zero true positives, both values are 0.0 and the denominator is 0, crashing the evaluation pipeline.

Prior review overlap:
Gemini flagged (in PR #402): NaN values from motmetrics (when metrics are undefined) are not handled. Gemini proposed: denominator = precision + recall; f1_score = 2 * precision * recall / denominator if denominator > 0 else 0.0. Gemini's fix is actually sufficient for NaN: in Python, float('nan') > 0 evaluates to False, so the else 0.0 branch fires for NaN inputs too.

New contribution in this analysis:
This analysis extends beyond that by: tracing where the NaN originates — motmetrics returns NaN for precision and recall in precision.py, recall.py, mota.py, motp.py, and idf1.py, all of which use the same unguarded round(float(summary.iloc[-1][...]), 4) pattern. Gemini's fix only patches f1_score.py; the same guard is needed in all five upstream scripts to prevent NaN from silently propagating into the leaderboard CSV. A partial fix to f1_score.py alone leaves NaN values in the precision, recall, mota, motp, and idf1 leaderboard columns.

Challenge:
The ZeroDivisionError surfaces at test time, not at configuration time. Any regressed model (or a correctly-configured model on an incorrect dataset) will hit this and crash CI.

Motivation:
Unguarded division in a metric function breaks CI for all future algorithm submissions. It also makes ianvs less reliable as a benchmarking platform.

Linked PRs: #402 (open)

Additional insights:
A robust fix should be applied to all six tracking metric scripts. The minimal guard if (denom := precision + recall) > 0 else 0.0 (Python 3.8+ walrus operator) handles both zero and NaN cases simultaneously.

---

Issue #93 — 'MultiedgeInference' object has no attribute 'modules_funcs'

Introduction:
Reported February 2024, this unresolved bug crashes the ianvs pipeline for any multiedgeinference job: (paradigm=multiedgeinference) pipeline runs failed, error: 'MultiedgeInference' object has no attribute 'modules_funcs'. The MultiedgeInference class inherits from ParadigmBase but does not initialize modules_funcs, which the Sedna framework accesses during pipeline execution via build_paradigm_job().

Challenge:
Core framework bug, invisible at configuration time. Reproducing it requires a working Sedna installation and a GPU to launch the job. It has no linked PR after two years, suggesting the interface contract between ianvs and Sedna is underdocumented.

Motivation:
multiedgeinference is exercised by only one example in ianvs (this MOT17 example), so this bug went undetected for over two years. Fixing it is essential to validate the paradigm. Note: a different crash (SA-13: missing onnx) surfaces first on most machines, masking this bug.

Linked PRs: None.

Additional insights:
Workflow to isolate the bug: first install ONNX (eliminates SA-13 crash), then run a minimal multiedgeinference job. The modules_funcs error will appear in the pipeline traceback.

---

Issue #335 — Incorrect logical operator in _check_fields validation

Introduction:
_check_fields() methods in four core modules use and instead of or in validation guards. An empty string "" passes validation silently because not "" and not isinstance("", str) evaluates to True and False = False — no error is raised.

Prior review overlap:
Gemini flagged (in PR #336): Five specific locations where the error messages are misleading after the and → or change (e.g., "must be provided and be string type" is incorrect when using or). Gemini recommended updating error messages to "must be a non-empty string/dict/list" or splitting into separate checks.

New contribution in this analysis:
This analysis extends beyond that by: connecting the validation bug to a concrete MOT17 failure mode: when a user provides a YAML with a blank url field (common when copying stale documentation), the current code passes validation silently and then fails deep in the pipeline with a FileNotFoundError. After PR #336 is merged, that user will instead receive an immediate ValueError at config-parse time. This specificity is not present in either the issue or gemini's review.

Motivation:
Affects all ianvs examples. For MOT17 specifically, better validation errors reduce mean-time-to-fix from minutes (tracing deep pipeline errors) to seconds (instant ValueError on startup).

Linked PRs: #336 (open)

---

Issue #14 — initial_model_url vs base_model_url naming inconsistency

Introduction:
multiedge_inference.py line 82 sets os.environ["BASE_MODEL_URL"] = trained_model, using the old environment variable name. Both MOT17 algorithm YAML files use initial_model_url as the key name, which is correctly read into self.initial_model via kwargs.get("initial_model_url") (line 55).

Latent/indirect impact on MOT17: After careful code inspection (see issue14_verification.md), the current MOT17 _inference() code path passes the model URL directly as a parameter to job.load(trained_model). The ByteTrack basemodel.load(model_url=None) method uses the passed parameter, not the environment variable. Therefore the naming inconsistency does not cause an immediate failure in the current MOT17 execution path. However, the impact is latent: (a) any Sedna framework internals or monitoring tools that read BASE_MODEL_URL will see an incorrectly-named variable; (b) any future BaseModel that uses the env var bridge instead of the direct parameter will silently fail. PR #309's fix is still warranted for hygiene and forward compatibility.

Challenge:
Because the current MOT17 inference path bypasses the environment variable bridge entirely, the bug produces no observable error, making it very difficult to detect through testing alone. Confirming the true impact requires tracing two separate call paths — the ianvs multiedge_inference.py side and the Sedna SDK internals — to establish exactly when BASE_MODEL_URL is read versus when the direct parameter is used. The inconsistency only becomes a hard failure when a BaseModel implementation follows the Sedna env-var pattern, which is not the case today but is documented as the expected integration pattern.

Motivation:
Low urgency for MOT17 today, but represents technical debt that will silently break any new algorithm that follows the Sedna env-var pattern for model loading rather than the direct-parameter pattern.

Linked PRs: #309 (open)

---

Issue #132 — Complete unclear dependencies for Ianvs

Introduction:
The MOT17 example has no requirements.txt file. Users must manually piece together dependencies from the README's scattered pip install lines, which omit loguru, motmetrics, mmcv, and opencv-python.

Challenge:
Issue #231 (Term 3 2025) already identified cv2 and mmcv as missing. PR #229 added a requirements.txt but was closed without merge. The current main branch has no requirements file, meaning every new user must re-discover missing packages through trial-and-error ImportError cycles.

Motivation:
Without version-pinned dependencies, motmetrics version incompatibilities (1.x vs 0.x API differences) and pandas deprecation warnings add noise to first-time runs.

Linked PRs: #229 (closed without merge — contained requirements.txt)

Additional insights:
Minimal requirements.txt for current MOT17 example:

motmetrics>=1.2.0
loguru
fpdf
pandas>=1.3.0
seaborn
scikit-learn
scipy
opencv-python
mmcv

---

Issue #231 — PRE-TASK: MOT17 Multi-Edge Inference Benchmark (LFX TERM 3 2025)

Introduction:
Closed pre-task from LFX Term 3 2025 (NishantSinghhhhh). Identified eight execution-blocking problems: YAML path mismatches, dataset conversion script issues, missing cv2/mmcv, workspace directory creation, and documentation inconsistencies. PR #229 was opened to address them but closed without merge on November 22, 2025.

Why PR #229 was closed — exact maintainer feedback:

• MooreZheng (2025-07-29): "The current challenge and solution are not super clear for reviewers. A few suggestions: 1. Establish an issue to describe what is wrong with the current MOT17 example. 2. Link this PR to the corresponding issue and point out how this PR solves the problems."
• hsj576 (2025-08-05): "Overall, this PR looks fine to me, but the author needs to merge the commits into a single commit."
• MooreZheng (2025-08-05): "Overall, it looks fine. A few suggestions: 1. Use Python logger instead of print. 2. Use Kaggle, rather than the Ianvs Github repo, to store the PDF and CSV."

The author made additional commits addressing file sizes and documentation, but the PR was ultimately closed without being re-reviewed or merged.

Motivation:
The failure of PR #229 means all eight issues from #231 remain open in main as of 2026-05-21. Four of those issues were re-filed as new bugs in April 2026 (#391, #397, #399, #401) — demonstrating that without a merged fix, users will continue rediscovering the same bugs indefinitely.

Linked PRs: #229 (closed)

Additional insights:
Any new restoration PR must follow the maintainer guidance: single squashed commit, issue linked, Python logging module instead of print(), large files on Kaggle instead of the repo. A Kaggle mirror created by NishantSinghhhhh is already available: Kaggle MOT17 Dataset.

---

4. Original Findings: Static Analysis of main Branch (2026-05-21)

The following issues were discovered through static analysis of the current main branch and have since been filed as GitHub issues (#443–#450 and a comment on #391). Each represents an independent execution failure or latent code bug not present in any prior issue, PR, or automated review.

---

SA-1 — cmc.py line 48: hardcoded stale output path — comment on #391

File: testenv/reid/cmc.py:48
Code: output_dir = Path("./examples/pedestrian_tracking/multiedge_inference_bench/cmc")

Introduction: The CMC curve plot is saved to a stale path that references the pre-rename directory structure. generate_reports.py line 64 embeds this file path in the PDF via self.image(reid_result["cmc"], ...), so if the file is not present at the expected location, PDF generation fails. This path is distinct from the YAML metric URL paths covered by PR #392/#398 — it resides inside a Python script, not a YAML config. Gemini flagged this in PR #392 and it was marked as resolved in the PR branch; however, the path remains broken in main until the PR is merged.

Challenge: The stale path is in a Python script at a different directory depth than the corresponding YAML configs, so a global YAML find-and-replace will miss it. Reviewers must audit Python scripts separately from YAML files.

Motivation: Without this fix, Step 3 of the MOT17 workflow (PDF report generation) fails even after both benchmark jobs succeed, leaving users with no final output despite a full GPU run.

Linked PRs: None (PR #392 branch may contain a fix but is unmerged)
Filed as: Comment on #391

Additional insights: Fix: change line 48 to output_dir = Path("./examples/MOT17/multiedge_inference_bench/pedestrian_tracking/cmc"). SA-1 is tracked separately from #391 because the stale path occurs inside executable Python logic rather than YAML configuration files, requiring an independent code change in a different file.

---

[#443] ByteTrack basemodel.py: DDP and dist never imported (SA-2 & SA-3)

File: testalgorithms/tracking/byte_track/basemodel.py

• Line 115: self.model = DDP(self.model, device_ids=[self.rank])
• Line 176: batch_size = batch_size // dist.get_world_size()

Introduction: Neither torch.nn.parallel.DistributedDataParallel (aliased as DDP) nor torch.distributed (aliased as dist) are imported anywhere in the file. In single-GPU mode these lines are unreachable (is_distributed = num_gpu > 1), so the bug is invisible in the most common single-GPU setup. On any multi-GPU server — the primary intended deployment for an edge inference benchmark — both lines raise NameError the moment distributed model loading or data loading is attempted.

Challenge: The bug is silent on the most common developer hardware (single GPU or CPU). It surfaces only on multi-GPU machines, which are also the machines most likely to be used in production edge deployments. This makes it easy to miss in standard CI.

Motivation: The multiedgeinference paradigm is specifically designed for distributed edge scenarios. Missing imports for distributed primitives undermine the entire design goal of the example.

Linked PRs: None
Filed as: #443

Additional insights: Fix — add to basemodel.py imports:

import torch.distributed as dist
from torch.nn.parallel import DistributedDataParallel as DDP

---

[#444] ByteTrack basemodel.py: tautological assertion and malformed YAML indentation (SA-4 & SA-5)

Filed as: #444

SA-4 — Tautological assertion (basemodel.py line ~86)

File: testalgorithms/tracking/byte_track/basemodel.py

num_gpu = torch.cuda.device_count()          # line N
assert num_gpu <= torch.cuda.device_count()  # line N+1

Introduction: Verified against the main branch: num_gpu is assigned from torch.cuda.device_count() on the immediately preceding line, then the assertion checks that same value against the identical function call. The condition n <= n is always True, so the assertion never fires under any circumstances.

Challenge: On a CPU-only machine (num_gpu = 0), the assertion passes silently, deferring the failure to the .cuda() call further down with a less descriptive RuntimeError: CUDA is not available. The tautology masks the intended early-exit guard, making it harder to diagnose environment issues.

Motivation: An effective assertion here would provide a clear, immediate message for users without CUDA, rather than a confusing error several lines later. The intent was likely assert num_gpu >= 1, "No CUDA GPU found".

Linked PRs: None

Additional insights: The fix is a one-line change. Confirm CUDA availability before reaching .cuda() calls reduces user confusion on CPU-only or misconfigured machines.

SA-5 — Malformed YAML indentation (byte_track_algorithm.yaml)

File: testalgorithms/tracking/byte_track/byte_track_algorithm.yaml:25-27

      hyperparameters:
        # name of the hyperparameter; string type;
          - batch_size:          # ← 10-space indent
             values:
              - 1

Introduction: The - batch_size: list item uses 10 spaces of indentation; the sibling m3l_algorithm.yaml uses the standard 8 spaces. The extra indentation may be interpreted inconsistently depending on parser/version behavior, potentially raising yaml.scanner.ScannerError or silently dropping the batch_size hyperparameter.

Challenge: Silent failures are especially hard to detect here — if PyYAML parses without raising an error, the algorithm proceeds with an unintended default batch size, producing different results than documented without any visible warning.

Motivation: Hyperparameter misconfiguration affects reproducibility. If batch_size defaults silently to 1 instead of the intended value, benchmark results cannot be compared against documented baselines.

Linked PRs: None

Additional insights: Fix: align - batch_size: to 8-space indent to match m3l_algorithm.yaml. Validate with python -c "import yaml; yaml.safe_load(open('byte_track_algorithm.yaml'))" before committing.

---

[#445] M3L basemodel.py: unguarded regex and deprecated addmm_ API (SA-6 & SA-7)

Filed as: #445

SA-6 — Regex without None guard (basemodel.py line 56)

File: testalgorithms/reid/m3l/basemodel.py:56
Code: arch = re.compile("_([a-zA-Z]+).pth").search(model_url).group(1)

Introduction: If model_url does not match the pattern _<letters>.pth (e.g., a filename like market_best_model.bin), re.search() returns None and .group(1) raises AttributeError: 'NoneType' object has no attribute 'group'. The period in the regex is also unescaped, so it matches any character, not just a literal dot. This is a latent bug — the current default filename market_IBNMeta.pth.tar happens to match, but any alternative checkpoint may fail.

Challenge: The failure occurs at model load time, before any inference runs, with a confusing AttributeError that gives no indication of which filename was the problem. Users experimenting with alternative checkpoints will hit this immediately.

Motivation: A benchmarking tool should accept model checkpoints with varied naming conventions. Hard-coding a filename pattern as an implicit contract prevents users from substituting their own fine-tuned models.

Linked PRs: None

Additional insights: Fix with a None guard and escaped regex:

match = re.compile(r"_([a-zA-Z]+)\.pth").search(model_url)
if not match:
    raise ValueError(f"Cannot infer architecture from model filename: {model_url}")
arch = match.group(1)

SA-7 — Deprecated addmm_ API (basemodel.py line 153)

File: testalgorithms/reid/m3l/basemodel.py:153
Code: dist_m.addmm_(1, -2, x, y.t())

Introduction: The three-positional-float signature addmm_(beta, alpha, mat1, mat2) was deprecated in PyTorch 1.5 and removed in PyTorch 2.0. On any modern GPU environment (CUDA 11.8+ typically ships with PyTorch ≥ 2.0), this raises TypeError: addmm_() received an invalid combination of arguments. The example was written in 2022 and this API change has silently broken it on any current GPU server.

Challenge: The error only surfaces during the pairwise distance computation inside predict(), which requires a full dataset pass through the model. Users must wait for inference to complete before discovering the crash, wasting significant compute time.

Motivation: PyTorch 2.0 is the standard in most 2024–2026 GPU environments. Leaving this unpatched means the ReID job cannot complete on any current server, blocking the entire MOT17 benchmark workflow.

Linked PRs: None

Additional insights: Fix — use keyword arguments consistent with modern PyTorch:

dist_m.addmm_(x, y.t(), beta=1, alpha=-2)

---

[#447] ReID metrics: IndexError and RuntimeError on edge case inputs (SA-8 & SA-9)

Filed as: #447

SA-8 — rank_1/2/5.py: IndexError when no gallery match

Files: testenv/reid/rank_1.py:31, rank_2.py:31, rank_5.py:31
Code: k = np.nonzero(matches[i])[0][0]

Introduction: For query person i, if no gallery image shares the same identity, np.nonzero(matches[i]) returns an empty array. Indexing it with [0][0] raises IndexError: index 0 is out of bounds for axis 0 with size 0. The loop iterates over all m queries with no guard, so a single unmatched query crashes the entire ReID metric evaluation.

Challenge: This condition can occur legitimately in properly-constructed datasets (e.g., a query person who only appears in the query split, not the gallery). It is not a data error but a valid edge case that the metric implementation must handle gracefully.

Motivation: A single unmatched query destroys the entire Rank-1/2/5 evaluation. Because all three metric files share the same pattern, the same fix must be applied to rank_1.py, rank_2.py, and rank_5.py independently.

Linked PRs: None

Additional insights: Fix — add a bounds guard inside the loop:

nonzero = np.nonzero(matches[i])[0]
if len(nonzero) == 0:
    continue
k = nonzero[0]

SA-9 — mAP.py line 38: unhandled RuntimeError on empty query set

File: testenv/reid/mAP.py:38
Code: raise RuntimeError("No valid query")

Introduction: If every query has no matching gallery identity, the aps list remains empty and RuntimeError("No valid query") propagates as an unhandled exception through the ianvs pipeline, surfacing only as a generic testcase runs failed error with no diagnostic information.

Challenge: The RuntimeError is indistinguishable at the ianvs level from other pipeline failures. Without reading the full traceback, users cannot tell whether the problem is with the dataset split, the model, or the metric code itself.

Motivation: A metric function that raises an unhandled exception breaks the benchmark more severely than returning a score of 0.0. Returning 0.0 on an empty query set is consistent with how rank_1.py skips unmatched queries, and preserves the pipeline's ability to continue and record partial results.

Linked PRs: None

Additional insights: Fix: replace raise RuntimeError(...) with return 0.0 and optionally log a warning using the Python logging module.

---

[#448] NaN propagation from 5 upstream tracking metrics (SA-10)

Filed as: #448

Files: testenv/tracking/precision.py, recall.py, mota.py, motp.py, idf1.py

Introduction: All five scripts share the unguarded pattern return round(float(summary.iloc[-1][metrics_name]), 4). When motmetrics cannot compute a metric (e.g., no detections at all), summary.iloc[-1][...] returns float('nan'). float(nan) succeeds and round(nan, 4) returns nan, which is then written to the leaderboard CSV.

Challenge: PR #402's fix for f1_score.py (if denominator > 0 else 0.0) handles the NaN case correctly in isolation — in Python, nan > 0 evaluates to False, so the guard fires. However, the precision, recall, mota, motp, and idf1 columns in the leaderboard CSV will still contain nan until those five upstream scripts are also patched. A partial fix to f1_score.py alone produces a leaderboard with mixed valid and NaN entries.

Motivation: NaN values in the leaderboard CSV corrupt downstream analysis and comparisons. A complete and consistent fix requires applying NaN sanitization to all six metric scripts uniformly.

Linked PRs: None (PR #402 partially addresses the downstream effect in f1_score.py)

Additional insights: Apply to all five scripts: value = float(summary.iloc[-1][...]) then return 0.0 if (math.isnan(value) or math.isinf(value)) else round(value, 4).

---

[#449] Missing requirements.txt and Docker-specific workspace path (SA-11 & SA-12)

Filed as: #449

SA-11 — No requirements.txt

Path: examples/MOT17/multiedge_inference_bench/pedestrian_tracking/

Introduction: No requirements.txt exists in this directory. Users must piece together dependencies from the README's scattered pip install commands, which omit loguru, motmetrics, mmcv, and opencv-python. This is the first obstacle for any new contributor attempting to set up the environment.

Challenge: Without version pinning, dependency resolution is non-deterministic. motmetrics 1.x and 0.x have incompatible metric naming conventions; an unconstrained pip install motmetrics may install either version. The same applies to pandas, where delim_whitespace=True was deprecated in 2.0.

Motivation: The absence of a requirements.txt blocks any automated CI setup and forces every new contributor to manually debug import errors before running a single line of benchmark code.

Linked PRs: None (PR #229 contained a requirements.txt but was closed without merge)

Additional insights: Minimal requirements.txt:

motmetrics>=1.2.0
loguru
fpdf
pandas>=1.3.0,<2.0
seaborn
scikit-learn
scipy
opencv-python
mmcv

SA-12 — Workspace path is Docker-specific

Files: tracking_job.yaml:5, reid_job.yaml:5
Code: workspace: "/ianvs/multiedge_inference_bench/workspace"

Introduction: The workspace path /ianvs/multiedge_inference_bench/workspace is an absolute path that exists only inside the official ianvs Docker container. Users running ianvs in a virtualenv or Conda environment (the recommended bare-metal setup per the ianvs docs) must manually create this root-owned directory, or the benchmark fails with PermissionError. The README does not mention this requirement.

Challenge: The failure is not obvious: the error occurs at the start of the benchmark run, before any inference begins, with a PermissionError or FileNotFoundError that does not mention the Docker context. Users on bare-metal setups have no way to know this path is Docker-specific from the error message alone.

Motivation: Bare-metal and virtualenv setups are the most common developer environments. Hardcoding a Docker-only path as the default blocks the majority of contributors from running the example without additional undocumented setup steps.

Linked PRs: None

Additional insights: Fix: change to a relative path consistent with other ianvs examples: workspace: "./workspace/multiedge_inference_bench"

---

[#450] multiedge_inference.py: unconditional import onnx (SA-13)

Filed as: #450

File: core/testcasecontroller/algorithm/paradigm/multiedge_inference/multiedge_inference.py:20
Code: import onnx

Introduction: onnx is imported unconditionally at module load time, but onnx is a heavy dependency not included in the core ianvs requirements.txt. The MOT17 example uses the _inference() code path, not _inference_mp(), so it never calls _partition() and has no need for ONNX. Despite this, any attempt to run the MOT17 example fails with ModuleNotFoundError: No module named 'onnx' before a single line of user code executes.

Challenge: The error message gives no indication that ONNX is only needed for model partitioning. Users must read the source code of multiedge_inference.py to understand why ONNX is required for an example that does not use partitioning. This is also a core framework file, so the fix must be made carefully to avoid breaking other paradigm uses.

Motivation: ONNX is an optional heavy dependency (it brings in many transitive dependencies). Making it a hard requirement for all multiedgeinference users — including those who only use the direct inference path — unnecessarily increases installation complexity. This also masks Issue #93 (modules_funcs AttributeError), which is the next error users would encounter if ONNX were installed.

Linked PRs: None

Additional insights: Fix — convert to a lazy import inside _partition():

def _partition(self, partition_point_list, initial_model_path, sub_model_dir):
    import onnx  # lazy import; only needed for model partitioning
    ...

---

5. Summary

The MOT17/multiedge_inference_bench example is blocked by 18 distinct GitHub issues (covering 22 individual execution blockers across 13 original static-analysis findings and 9 tracked prior issues) across the following categories.

Category	Count	Issues
Path / Configuration	4	#391, #397, SA-1(2)
Runtime Errors / Crashes	4	#399, #401, #447, #448
Algorithm Code Bugs	3	#443, #444, #445
Naming / API Bug	1	#14
Core Framework	3	#93, #335, #450
Missing Dependencies / Docs	2	#132, #449
Historical / Prior Work	1	#231
Total	18

Several findings are causally related but counted separately because they require independent code changes in different files.

A complete restoration requires addressing all categories in order:
Framework (#93, #450) → Paths (#391, #397, SA-1, #449) → Algorithm code (#443, #444, #445, #14) → Runtime errors (#399, #401, #447, #448) → Dependencies (#132, #449)

[suhaan-24]

I reviewed all open PRs that touch the MOT17/multiedge_inference_bench example (and two related core PRs) as part of a broader effort to make this example runnable end-to-end. Below is a summary of all line-level findings across the seven PRs, followed by a merge-ordering recommendation.

Each PR was reviewed using GitHub's integration mode — a main review comment plus line-specific sub-comments pinpointing exact locations in the diff. The main review comment for each PR is the authoritative source of findings and includes a brief summary of all sub-comment issues. Direct links to every submitted review are in the PR Review Summary table below.

SPECIFIC GREEN CHANNEL

---

Verification via Automated Test Suite

This review is backed by a purpose-built, fully executable test suite
run against every PR branch independently — not just static diff reading.

To move beyond manual inspection, I built
test_mot17.py
— a 24-test suite designed specifically for MOT17/multiedge_inference_bench.
It runs without a GPU, without the dataset, and without model weights,
using mocks and static analysis to exercise the actual code paths.

Three test layers:

Layer	What it tests	# Tests
Layer 1 — Static	Stale path detection, workspace path safety, YAML metric URL correctness, pandas API compatibility, NameError detection, import availability	8
Layer 2 — Unit	ZeroDivisionError/NaN guards in all 6 metric scripts, save_mode regression, ONNX import scope, rank IndexError, mAP empty query	14
Layer 3 — Smoke	generate_reports.py dry run, workspace createability	2

I ran the suite against main and every PR branch independently:

# Example run against PR #392
git fetch upstream pull/392/head:pr-392
git checkout pr-392
python examples/MOT17/multiedge_inference_bench/test_mot17.py

Pipeline overview:

flowchart TD
    classDef baseline fill:#6b7280,stroke:#4b5563,color:#fff
    classDef improved fill:#16a34a,stroke:#15803d,color:#fff
    classDef regression fill:#dc2626,stroke:#b91c1c,color:#fff
    classDef closed fill:#9333ea,stroke:#7e22ce,color:#fff
    classDef runner fill:#2563eb,stroke:#1d4ed8,color:#fff
    classDef layer fill:#0891b2,stroke:#0e7490,color:#fff

    subgraph INPUT["📥 Input — 8 Branches"]
        main:::baseline
        PR392["PR #392"]:::improved
        PR398["PR #398"]:::improved
        PR400["PR #400"]:::improved
        PR402["PR #402"]:::improved
        PR336["PR #336"]:::regression
        PR309["PR #309"]:::baseline
        PR229["PR #229 · closed"]:::closed
    end

    RUNNER[["🧪 test_mot17.py · 24 tests"]]:::runner

    subgraph LAYERS["Test Layers"]
        L1["🔎 Layer 1 · Static · 8 tests\nStale paths · Workspace · YAML URLs · Pandas · NameError"]:::layer
        L2["⚙️ Layer 2 · Unit · 14 tests\nNaN guards · save_mode regression · ONNX · IndexError · mAP"]:::layer
        L3["💨 Layer 3 · Smoke · 2 tests\ngenerate_reports dry run · workspace createability"]:::layer
    end

    subgraph SCORES["📊 Results per Branch"]
        S_main["main · 7/24 · baseline"]:::baseline
        S392["#392 · 8/24 · ✅ +1"]:::improved
        S398["#398 · 9/24 · ✅ +2"]:::improved
        S400["#400 · 9/24 · ✅ +2"]:::improved
        S402["#402 · 8/24 · ✅ +1"]:::improved
        S336["#336 · 6/24 · ⚠️ −1 REGRESSION"]:::regression
        S309["#309 · 7/24 · = no change"]:::baseline
        S229["#229 · 10/24 · ✅ +3 · closed"]:::closed
    end

    INPUT --> RUNNER
    RUNNER --> L1 & L2 & L3
    L1 & L2 & L3 --> SCORES

Loading

Results matrix — every cell is a real test execution, not an estimate:

Test	main	PR #392	PR #398	PR #400	PR #402	PR #336	PR #309	PR #229
1.1 Stale path detector	❌	✅	❌	❌	❌	❌	❌	❌
1.2 Workspace path checker	❌	❌	✅	❌	❌	❌	❌	❌
1.3 YAML metric URL validator	✅	✅	✅	✅	✅	✅	✅	✅
1.4 Required files existence	✅	✅	✅	✅	✅	✅	✅	✅
1.5 Import dependency check	✅	✅	✅	✅	✅	✅	✅	✅
1.6 Pandas API compatibility	❌	❌	❌	❌	❌	❌	❌	✅
1.7 YAML indentation validator	✅	✅	✅	✅	✅	✅	✅	✅
1.8 NameError check	❌	❌	❌	✅	❌	❌	❌	✅
2.1 f1_score ZeroDivisionError	❌	❌	❌	❌	✅	❌	❌	❌
2.2 f1_score NaN guard	❌	❌	❌	❌	❌	❌	❌	❌
2.3 f1_score normal computation	✅	✅	✅	✅	✅	✅	✅	✅
2.4a NaN propagation (precision)	❌	❌	❌	❌	❌	❌	❌	❌
2.4b NaN propagation (recall)	❌	❌	❌	❌	❌	❌	❌	❌
2.4c NaN propagation (mota)	❌	❌	❌	❌	❌	❌	❌	❌
2.4d NaN propagation (motp)	❌	❌	❌	❌	❌	❌	❌	❌
2.4e NaN propagation (idf1)	❌	❌	❌	❌	❌	❌	❌	❌
2.5 generate_reports NameError	❌	❌	❌	✅	❌	❌	❌	✅
2.6 generate_reports None guard	❌	❌	❌	❌	❌	❌	❌	❌
2.7 rank_1 IndexError guard	❌	❌	❌	❌	❌	❌	❌	❌
2.8 mAP empty query	❌	❌	❌	❌	❌	❌	❌	❌
2.9 save_mode string regression	✅	✅	✅	✅	✅	❌	✅	✅
2.10 ONNX lazy import	❌	❌	❌	❌	❌	❌	❌	❌
3.1 generate_reports dry run	✅	✅	✅	✅	✅	✅	✅	✅
3.2 Workspace createability	❌	❌	✅	❌	❌	❌	❌	❌
TOTAL	7/24	8/24	9/24	9/24	8/24	6/24 ⚠️	7/24	10/24

Three findings confirmed directly by test execution:

• ✅ PR fix: replace 'and' with 'or' in _check_fields validation logic #336 scores below main (6/24 vs 7/24) — test 2.9 confirms the save_mode regression: the or fix causes ValueError for every existing string-valued config that was previously valid
• ✅ PR ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero #402's == 0 guard does not catch NaN — test 2.2 confirms: nan + nan == 0 evaluates to False in Python; the division still executes and returns nan
• ✅ PR Added files for MOT17 #229's sep=',' silently misparses whitespace-delimited output — identified via static analysis: ianvs ranking output is whitespace-delimited; the comma separator produces a one-column DataFrame with no exception raised

Execution proof (video, screenshots):

[https://drive.google.com/file/d/1UQ3OrUgocig5C530QV2XRoSwp53HABK0/view?usp=drive_link]

Test suite source:
feat/mot17-test-suite branch → examples/MOT17/multiedge_inference_bench/test_mot17.py

---

Background

The following table lists all 7 PRs reviewed, categorized by the type of change they introduce.

#	PR	Title	Category	Status
1	#392	Fix stale pedestrian_tracking path references	Bug Fix (Path)	Open
2	#398	Fix directory mismatches + workspace paths	Bug Fix (Path)	Open
3	#400	Fix NameError in generate_reports.py	Bug Fix (Runtime)	Open
4	#402	Fix ZeroDivisionError in f1_score.py	Bug Fix (Runtime)	Open
5	#336	Fix and→or in _check_fields validation	Bug Fix (Core)	Open
6	#309	Rename BASE_MODEL_URL → INITIAL_MODEL_URL	Refactor (Core)	Open
7	#229	Full MOT17 restoration	Full Restoration	Closed

---

PR Review Summary

PR	Title	Verdict	Critical?	Review Link
#392	Fix stale pedestrian_tracking path references	REQUEST CHANGES	No — 1 high issue	Review
#398	Fix directory mismatches + workspace paths	REQUEST CHANGES	No — 1 high issue	Review
#400	Fix NameError in generate_reports.py	REQUEST CHANGES	No — 2 high issues	Review
#402	Fix ZeroDivisionError in f1_score.py	REQUEST CHANGES	Yes — 1 blocker	Review
#336	Fix and→or in _check_fields validation	REQUEST CHANGES	Yes — 1 blocker	Review
#309	Rename BASE_MODEL_URL → INITIAL_MODEL_URL	APPROVE (with suggestions)	No	Review
#229	Full MOT17 restoration	CLOSED — documented below	Yes — 2 blockers	N/A (PR closed)

---

Detailed Summarization

PR #392 — Fix stale pedestrian_tracking path references — Review

Introduction: This PR addresses the most critical path regression in the MOT17 example: the pervasive stale pedestrian_tracking/ prefix across 9 files. The dynamic path resolution introduced for generate_reports.py and cmc.py is a good improvement over the original hardcoded strings. However, one issue must be addressed before this can be safely merged.

Module details:

• Scope: Example-only (MOT17)
• Files changed: 9 files under examples/MOT17/multiedge_inference_bench/pedestrian_tracking/ — README.md, generate_reports.py, reid_job.yaml, tracking_job.yaml, two algorithm YAMLs, cmc.py, and two testenv.yaml files
• Core changes: None — all changes are confined to the MOT17 example directory

---

PR #398 — Fix directory mismatches + workspace paths — Review

Introduction: This PR correctly fixes the workspace absolute path and path prefix mismatches in reid_job.yaml and tracking_job.yaml — these are the right changes for MOT17 bare-metal users. However, the PR is over-scoped: it bundles MOT17 path fixes with behavioral changes for at least 3 other examples, making it difficult to review or revert safely.

Module details:

• Scope: Multi-example (MOT17 + LLM example + others)
• Files changed: 20 files across at least 4 different examples
• Core changes: None — changes are confined to example directories; however, mixing examples in one PR creates review and bisect risk

---

PR #400 — Fix NameError in generate_reports.py — Review

Introduction: This PR correctly identifies the NameError on lines 77 and 86 where error messages referenced config_file instead of the locally-scoped variables tracking_config_file and reid_config_file. The new messages are more informative. However, two additional issues in the same file should be addressed since the PR already modifies this code.

Module details:

• Scope: Example-only (MOT17)
• Files changed: 1 file — examples/MOT17/multiedge_inference_bench/pedestrian_tracking/generate_reports.py
• Core changes: None

---

PR #402 — Fix ZeroDivisionError in f1_score.py — Review

Introduction: This PR correctly identifies the ZeroDivisionError in f1_score.py and adds a guard before the division. However, the guard condition == 0 is insufficient to handle NaN values, and the same unguarded edge case exists in five other tracking metric scripts that this PR does not address.

Module details:

• Scope: Example-only (MOT17)
• Files changed: 1 file — examples/MOT17/multiedge_inference_bench/pedestrian_tracking/testenv/tracking/f1_score.py
• Core changes: None — but 5 additional scripts in the same directory (precision.py, recall.py, mota.py, motp.py, idf1.py) have the same unresolved issue

---

PR #336 — Fix and→or in _check_fields validation — Review

Introduction: This PR changes and to or in the _check_fields validation across four core files. The intent is correct — the original and allowed an empty/non-list save_mode to slip through validation silently. However, the fix as written introduces a regression for all existing configs that pass save_mode as a string (the standard usage), and the error messages become inaccurate after the operator change.

Module details:

• Scope: Core
• Files changed: 4 files — core/cmd/obj/benchmarkingjob.py, core/storymanager/rank/rank.py, core/testcasecontroller/algorithm/algorithm.py, core/testcasecontroller/algorithm/module/module.py
• Core changes: Yes — affects all paradigms and all examples that use ianvs; the regression risk is high and the fix must be validated against the actual type of each field before merging

---

PR #309 — Rename BASE_MODEL_URL → INITIAL_MODEL_URL — Review

Introduction: The rename from BASE_MODEL_URL to INITIAL_MODEL_URL is applied consistently across the core paradigm files and all example basemodel.py files touched. The change is semantically correct — "initial" more accurately describes the pre-trained model checkpoint, and this is a clean, non-breaking rename within the codebase.

Module details:

• Scope: Core + Examples
• Files changed: 8 files — core/testcasecontroller/algorithm/paradigm/incremental_learning/incremental_learning.py, multiedge_inference.py, singletask_learning.py, and 5 example basemodel.py files
• Core changes: Yes — justified; the rename improves semantic clarity and the scope is consistent across all affected files. Minor risk: users with the old variable name in their shell or Docker environment will get a silent None without a deprecation fallback

---

PR #229 — Full MOT17 restoration (CLOSED)

Introduction: This PR attempted a full restoration of the MOT17 example. While the intent is correct, it has four technical issues that would prevent the example from running correctly even if the maintainer process concerns (issue requirement, squashed commits, logger vs print) were addressed.

Module details:

• Scope: Example-only (MOT17)
• Files changed: 6 files — README.md, docs/proposals/scenarios/multiple-object-tracking/mot_17.md (new), cmc_20230801100000.png (binary, 3.2 MB), generate_reports.py, reid_job.yaml, tracking_job.yaml
• Core changes: None — PR was closed by maintainer for process reasons; technical blockers documented in the findings table below for reference

---

All Line-Level Findings

PR	File	Line	Issue	Severity
#392	tracking_job.yaml, reid_job.yaml	5	workspace field not updated — still /ianvs/multiedge_inference_bench/workspace, which only exists inside Docker; bare-metal users get PermissionError immediately	High
#398	tracking_job.yaml, reid_job.yaml	4	./workspace resolves against CWD at invocation time, not relative to the YAML file; users who cd into the example directory will get a workspace in the wrong location; README should document the required working directory	High
#398	(all 20 files)	—	PR mixes MOT17 path fixes with LLM-example behavioral changes (device selection, division-by-zero guards); should be split into separate PRs per example so regressions can be isolated	Medium
#400	generate_reports.py	80	pd.read_csv(..., delim_whitespace=True) removed in pandas 2.0; raises TypeError: __init__() got an unexpected keyword argument 'delim_whitespace' on any current environment	High
#400	generate_reports.py	88	Same delim_whitespace=True deprecation on the ReID rank read; replace with sep=r'\s+'	High
#400	generate_reports.py	76	utils.is_local_file(None) raises TypeError if CLI flag is omitted; need a None guard before the is_local_file call (same at line 84 for reid_config_file)	Medium
#402	testenv/tracking/f1_score.py	53	if precision + recall == 0 does not catch NaN: nan + nan == 0 is False in Python, so the guard is skipped and nan / nan is returned. Fix: if not (precision + recall > 0)	Critical
#402	precision.py, recall.py, mota.py, motp.py, idf1.py	last line	Five other tracking metric scripts return raw motmetrics floats with no NaN guard; all five can silently corrupt the leaderboard CSV; this PR fixes only f1_score.py	High
#402	testenv/tracking/f1_score.py	35	mh.compute(..., disth=0.5) called without fmt= parameter; default is mot15-2D but MOT17 uses mot17-2D; wrong format means distance thresholds are applied to mis-interpreted coordinates	Medium
#336	core/storymanager/rank/rank.py	87	not isinstance(self.save_mode, list) is True for every existing config that uses save_mode: "selected_and_all" (a string); the or change raises ValueError unconditionally for all current benchmark jobs. The correct targeted fix is to check for None specifically: if self.save_mode is None: raise ValueError(...)	Critical
#336	rank.py (and 3 other core files)	89	Error messages do not list valid values; after a user hits the incorrect ValueError, they have no guidance on what save_mode should be	Medium
#309	multiedge_inference.py	82	Users with BASE_MODEL_URL set in their shell/Docker environment will silently get None after the rename; a transitional fallback (os.environ.get("INITIAL_MODEL_URL") or os.environ.get("BASE_MODEL_URL")) would ease migration	Medium
#309	README.md / docs/	—	Documentation references to BASE_MODEL_URL should be updated in this PR	Medium
#229	tracking_job.yaml, reid_job.yaml	4	../workspace/ resolves to the directory above the repo root when ianvs is invoked from the repo root; this is almost certainly a write-protected system directory	Critical
#229	generate_reports.py	80, 88	sep=',' replaces delim_whitespace=True, but ianvs ranking output is whitespace-delimited; the comma separator will produce a one-column DataFrame of unparsed rows with no error raised	High
#229	cmc_20230801100000.png	—	3.2 MB binary committed to git history; per maintainer feedback this should be hosted on Kaggle and referenced by URL	Low

---

Cross-PR Conflicts

File	Touched by	Risk
generate_reports.py	#392, #400, #229	Whoever merges last must rebase; the delim_whitespace fix in #400 and the dynamic path fix in #392 both modify nearby lines
tracking_job.yaml, reid_job.yaml	#392, #398, #229	#392 leaves workspace unchanged; #398 fixes it to ./workspace; both must land together or job YAMLs remain broken

---

Recommended Merge Order

1. #336 (core validation fix — but requires the save_mode type regression to be resolved first)
2. #309 (rename — no dependencies, low risk)
3. #398 (workspace + path fixes — unblocks bare-metal users; coordinate with fix: update stale pedestrian_tracking paths to fix broken examples #392)
4. #392 (path prefix fixes — coordinate workspace fix with fix: resolve file routing crashes by correcting MOT17 directory mismatches #398 before merge)
5. #402 (f1_score NaN guard — extend to all 5 other metric scripts first)
6. #400 (NameError fix — rebase on top of fix: update stale pedestrian_tracking paths to fix broken examples #392 after generate_reports.py conflicts resolve)

[mwemmz]

Reproduced independently by cloning main and running:

ianvs -f examples/MOT17/multiedge_inference_bench/pedestrian_tracking/tracking_job.yaml

Result:

RuntimeError: benchmarkingjob runs failed, error: not found testenv config file
(./examples/pedestrian_tracking/multiedge_inference_bench/testenv/tracking/testenv.yaml) in local.

Confirmed examples/pedestrian_tracking does not exist; current path is examples/MOT17/multiedge_inference_bench/pedestrian_tracking. Traced the failure to the identical core function (_parse_testenv_config, benchmarkingjob.py:117) as the equivalent bug in robot-cityscapes-synthia (#514) — both examples fail before any example-specific code executes.

Also fetched the live ci-managed/example-health-status branch (updated 2026-08-21) — both this example and robot-cityscapes-synthia are published with status "CI/CD onGoing", color: lightgrey, not a failing status, despite both being currently 100% non-runnable via the path shown above.