Skip to content

fix: hit-testing response failed because of Pydantic check.#35640

Merged
FFXN merged 18 commits intomainfrom
fix/hit-testing
Apr 28, 2026
Merged

fix: hit-testing response failed because of Pydantic check.#35640
FFXN merged 18 commits intomainfrom
fix/hit-testing

Conversation

@FFXN
Copy link
Copy Markdown
Contributor

@FFXN FFXN commented Apr 28, 2026

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

This PR fixes a 400 invalid_param regression in the dataset hit-testing API introduced after the response layer was tightened with Pydantic validation. The underlying retrieval flow could still return the legacy query={"content": ...} shape, and some optional collection fields such as segment.keywords, child_chunks, and files could be None, which no longer matched the current response schema expecting a str and concrete list values.

The change adds a small compatibility normalization step in the shared DatasetsHitTestingBase response path without altering retrieval behavior, ranking, scoring, or filtering. It converts the legacy query object into a plain string and normalizes nullable collection fields to empty lists before response validation. Targeted tests were also added to cover the legacy query shape and nullable list field cases, ensuring both console and service API hit-testing endpoints return schema-compliant responses consistently.

Screenshots

Before After
... ...

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint && make type-check (backend) and cd web && pnpm exec vp staged (frontend) to appease the lint gods

FFXN and others added 18 commits March 3, 2026 14:31
…m service including remote template service and database, return responding error message.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…m service including remote template service and database, return responding error message.
# Conflicts:
#	api/services/rag_pipeline/pipeline_template/remote/remote_retrieval.py
# Conflicts:
#	api/providers/vdb/vdb-weaviate/src/dify_vdb_weaviate/weaviate_vector.py
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Apr 28, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 38.66% 38.66% -0.01%
Strict coverage 38.19% 38.18% -0.01%
Typed symbols 19,391 19,391 0
Untyped symbols 31,147 31,156 +9
Modules 2537 2537 0

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

❌ Patch coverage is 64.51613% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.68%. Comparing base (1d3498f) to head (e62a67c).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...i/controllers/console/datasets/hit_testing_base.py 64.51% 3 Missing and 8 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #35640      +/-   ##
==========================================
- Coverage   85.69%   85.68%   -0.01%     
==========================================
  Files        4459     4459              
  Lines      208961   208991      +30     
  Branches    39068    39078      +10     
==========================================
+ Hits       179061   179078      +17     
- Misses      26743    26748       +5     
- Partials     3157     3165       +8     
Flag Coverage Δ
api 85.08% <64.51%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 28, 2026
@FFXN FFXN added this pull request to the merge queue Apr 28, 2026
Merged via the queue into main with commit 38eb04d Apr 28, 2026
35 of 36 checks passed
@FFXN FFXN deleted the fix/hit-testing branch April 28, 2026 08:46
@PHclaw
Copy link
Copy Markdown

PHclaw commented Apr 28, 2026

For workflow execution timeout issues:

The default timeout for workflow execution is often too short. Increase it in the configuration:

# docker-compose.yaml
services:
  api:
    environment:
      WORKFLOW_EXECUTION_TIMEOUT: 3600  # 1 hour instead of default 5 minutes

Also implement async timeout handling:

import asyncio

async def run_workflow(workflow_id, inputs, timeout=3600):
    try:
        result = await asyncio.wait_for(
            execute_workflow(workflow_id, inputs),
            timeout=timeout
        )
        return result
    except asyncio.TimeoutError:
        # Save partial state and return error
        partial_state = await save_partial_state(workflow_id)
        return {"error": "timeout", "partial_state": partial_state}

@PHclaw
Copy link
Copy Markdown

PHclaw commented Apr 29, 2026

For the workflow node timeout:

class WorkflowNode:
    timeout: int = 300

    async def execute(self, inputs):
        try:
            return await asyncio.wait_for(
                self._run(inputs), timeout=self.timeout
            )
        except asyncio.TimeoutError:
            return {'status': 'timeout', 'node': self.name}

fatelei pushed a commit that referenced this pull request Apr 29, 2026
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants