feat: add profile search api and tests.

[hjxilinx]

This pull request introduces a new profile searching algorithm to the analytics toolkit, providing a robust implementation for searching time series profiles using DTW (Dynamic Time Warping) and cosine similarity. It also adds a new API endpoint for profile searching , comprehensive input validation, and a suite of unit tests to ensure correctness. Additional improvements include a new Pearson correlation endpoint and enhancements to existing batch processing routes.

New profile searching algorithm and API:

• Added do_profile_search_impl in profile_search.py implementing core logic for searching a source time series against multiple target profiles using DTW or cosine similarity, with support for normalization, sliding window, and flexible result filtering.
• Introduced a new API endpoint /api/v1/analysis/profile-search in app.py that exposes the profile searching functionality, including input validation, error handling, and logging.

Testing and validation:

• Added comprehensive unit tests for the profile searching logic in unit_test.py, covering DTW and cosine similarity, normalization options, windowing, thresholding, error cases, and input validation.
  API and batch processing enhancements:

• Added a new /api/v1/analysis/pearsonr endpoint for Pearson correlation, with robust input and error handling.

• Updated batch processing to support both legacy and versioned endpoints, improving backward compatibility.

These changes significantly enhance the toolkit's analytical capabilities, robustness, and test coverage.# Description

Issue(s)

• https://project.feishu.cn/taosdata_td/feature/detail/6927903169

Checklist

Please check the items in the checklist if applicable.

• Is the user manual updated?
• Are the test cases passed and automated?
• Is there no significant decrease in test coverage?

[gemini-code-assist]

Code Review

This pull request introduces a profile matching feature using Dynamic Time Warping (DTW) and Cosine similarity, alongside a new Pearson correlation endpoint. The changes include the core algorithm implementation, new API routes for versioned analysis, and unit tests. The review feedback identifies several areas for improvement: potential runtime errors in the Pearson correlation endpoint due to missing data or NaN results, performance inefficiencies caused by redundant data validation and large sliding window generation, and a logic issue where sequences are incorrectly truncated during cosine similarity calculations.

[Copilot]

Pull request overview

Adds new analytics capabilities to TDgpt/taosanalytics by introducing a profile-matching algorithm (DTW/cosine) and exposing it via a new REST endpoint, alongside a new Pearson correlation endpoint and accompanying unit tests.

Changes:

• Added do_profile_match_impl (DTW/cosine similarity, normalization, windowing, top-N/threshold filtering).
• Exposed new endpoints: /api/v1/analysis/profile-match and /api/v1/analysis/pearsonr; added a versioned batch route alias.
• Added unit tests covering profile matching behavior and validation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
tools/tdgpt/taosanalytics/algo/tool/profile_match.py	Implements profile matching core logic with validation, normalization, and DTW/cosine scoring.
tools/tdgpt/taosanalytics/app.py	Adds new v1 endpoints for pearson correlation and profile matching; adds versioned batch route alias.
tools/tdgpt/taosanalytics/test/unit_test.py	Adds unit tests for do_profile_match_impl covering key scenarios and validation errors.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• download-r2.pytorch.org
  • Triggering command: /usr/bin/pip pip install -r /home/REDACTED/work/TDengine/TDengine/tools/tdgpt/requirements.txt -q (dns block)
  • Triggering command: /usr/bin/pip pip install -r /home/REDACTED/work/TDengine/TDengine/tools/tdgpt/requirements_ess.txt -q (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[hjxilinx]

@copilot extract method for validating min_window/max_widow, method for prifle_list formation, method for vaildating source_arr.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

@copilot extract method for validating min_window/max_widow, method for prifle_list formation, method for vaildating source_arr.

Already done in commit 9a591df. The three helpers are present in profile_search.py: _validate_source_data, _validate_min_max_window, and _validate_profile_list. The parent _validate_params orchestrates them.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zitsen]

Code Review

Critical Issue: Misleading API docstring in app.py → do_profile_search

The docstring example shows ts as a flat list while data is a 2D profile list — this is an invalid combination that will mislead API consumers:

// ❌ Shown in docstring (invalid)
"target_data": {
    "ts": [1, 2, 3, 4, 5, 6],
    "data": [ [1, 2, 3, 4, 5] ]
}

When data is a list of profiles (2D), ts must be a list of [start_ts, end_ts] pairs:

// ✅ Correct format
"target_data": {
    "ts": [[1, 5]],
    "data": [ [1, 2, 3, 4, 5] ]
}

Please fix the docstring example before merging.

[zitsen]

Code Review

Critical Issue: Misleading API docstring in app.py → do_profile_search

The docstring example shows ts as a flat list while data is a 2D profile list — this is an invalid combination that will mislead API consumers:

// ❌ Shown in docstring (invalid)
"target_data": {
    "ts": [1, 2, 3, 4, 5, 6],
    "data": [ [1, 2, 3, 4, 5] ]
}

When data is a list of profiles (2D), ts must be a list of [start_ts, end_ts] pairs:

// ✅ Correct format
"target_data": {
    "ts": [[1, 5]],
    "data": [ [1, 2, 3, 4, 5] ]
}

Please fix the docstring example before merging.

Misunderstood, approve all.