Skip to content

🎨 Enhance detection of stalled streamed results #8685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
giancarloromeo merged 7 commits into from
Dec 9, 2025
Merged

🎨 Enhance detection of stalled streamed results #8685

giancarloromeo merged 7 commits into from
Dec 9, 2025
+41 −10

Conversation

@giancarloromeo
Copy link
Contributor

@giancarloromeo giancarloromeo commented Dec 9, 2025 •
edited
Loading

What do these changes do?

This PR enhances the detection of stalled streamed results by introducing a mechanism to track when tasks make progress even when no items are yielded. The change addresses an issue where long-running search operations without yielded results could be incorrectly flagged as stalled.

Related issue/s

How to test

Dev-ops

@giancarloromeo giancarloromeo added this to the Delirium Noel milestone Dec 9, 2025
@giancarloromeo giancarloromeo self-assigned this Dec 9, 2025

This comment was marked as resolved.

Sign in to view

@codecov
Copy link

codecov bot commented Dec 9, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 30.00000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.26%. Comparing base (2aab05d) to head (9805c9b).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #8685      +/-   ##
==========================================
- Coverage   87.55%   87.26%   -0.30%     
==========================================
  Files        2020     1592     -428     
  Lines       79318    66476   -12842     
  Branches     1387      701     -686     
==========================================
- Hits        69450    58008   -11442     
+ Misses       9472     8235    -1237     
+ Partials      396      233     -163
Flag Coverage Δ
integrationtests 63.86% <ø> (-0.02%) ⬇️
unittests 85.87% <30.00%> (-0.52%) ⬇️
Components Coverage Δ
pkg_aws_library ∅ <ø> (∅)
pkg_celery_library 81.31% <33.33%> (-0.88%) ⬇️
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library 71.36% <ø> (ø)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 84.45% <ø> (-0.14%) ⬇️
agent 93.44% <ø> (ø)
api_server 91.37% <ø> (ø)
autoscaling 95.71% <ø> (ø)
catalog 92.06% <ø> (ø)
clusters_keeper 99.14% <ø> (ø)
dask_sidecar 91.72% <ø> (ø)
datcore_adapter 97.95% <ø> (ø)
director 75.72% <ø> (-0.09%) ⬇️
director_v2 91.29% <ø> (+0.02%) ⬆️
dynamic_scheduler 96.66% <ø> (ø)
dynamic_sidecar 90.82% <ø> (ø)
efs_guardian 89.83% <ø> (ø)
invitations 90.90% <ø> (ø)
payments 92.70% <ø> (ø)
resource_usage_tracker 92.00% <ø> (-0.16%) ⬇️
storage 86.58% <0.00%> (+0.08%) ⬆️
webclient ∅ <ø> (∅)
webserver 86.67% <ø> (-0.04%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2aab05d...9805c9b. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@giancarloromeo giancarloromeo added a:celery-library a:storage issue related to storage service labels Dec 9, 2025
@giancarloromeo giancarloromeo marked this pull request as ready for review December 9, 2025 10:09
@mergify
Copy link
Contributor

mergify bot commented Dec 9, 2025 •
edited
Loading

🧪 CI Insights

Here's what we observed from your CI run for 9805c9b.

✅ Passed Jobs With Interesting Signals

Pipeline Job Signal Health on master Retries 🔍 CI Insights 📄 Logs
CI unit-tests Base branch is healthy, but retries were needed. Could be early signs of flakiness 👀 Healthy 1 View View

odeimaiz
odeimaiz approved these changes Dec 9, 2025
View reviewed changes
Copy link
Member

@odeimaiz odeimaiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@giancarloromeo
Copy link
Contributor Author

giancarloromeo commented Dec 9, 2025

@mergify queue

@mergify
Copy link
Contributor

mergify bot commented Dec 9, 2025 •
edited
Loading

queue

🟠 Waiting for conditions to match

Details
  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
      • #approved-reviews-by>=2
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved=0
      • -conflict
      • -draft
      • base=master
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=🤖-do-not-merge
      • label=🤖-automerge
      • any of: [🛡 GitHub branch protection]
        • check-skipped = deploy to dockerhub
        • check-neutral = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of: [🛡 GitHub branch protection]
        • check-success = system-tests
        • check-neutral = system-tests
        • check-skipped = system-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = unit-tests
        • check-neutral = unit-tests
        • check-skipped = unit-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = check OAS' are up to date
        • check-neutral = check OAS' are up to date
        • check-skipped = check OAS' are up to date
      • any of: [🛡 GitHub branch protection]
        • check-success = integration-tests
        • check-neutral = integration-tests
        • check-skipped = integration-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = build-test-images (frontend) / build-test-images
        • check-neutral = build-test-images (frontend) / build-test-images
        • check-skipped = build-test-images (frontend) / build-test-images
      • any of: [🛡 GitHub branch protection]
        • check-success = SonarCloud Code Analysis
        • check-neutral = SonarCloud Code Analysis
        • check-skipped = SonarCloud Code Analysis

@mergify
Copy link
Contributor

mergify bot commented Dec 9, 2025 •
edited
Loading

Merge Queue Status

🟠 Waiting for queue conditions

Required conditions to enter a queue
  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of [📌 queue -> configuration change requirements]:
    • -mergify-configuration-changed
    • check-success = Configuration changed
  • any of [🔀 queue conditions]:
    • all of [📌 queue conditions of queue default]:
      • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
      • #approved-reviews-by>=2
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved=0
      • -conflict
      • -draft
      • base=master
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=🤖-do-not-merge
      • label=🤖-automerge
      • any of [🛡 GitHub branch protection]:
        • check-skipped = deploy to dockerhub
        • check-neutral = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of [🛡 GitHub branch protection]:
        • check-success = system-tests
        • check-neutral = system-tests
        • check-skipped = system-tests
      • any of [🛡 GitHub branch protection]:
        • check-success = unit-tests
        • check-neutral = unit-tests
        • check-skipped = unit-tests
      • any of [🛡 GitHub branch protection]:
        • check-success = check OAS' are up to date
        • check-neutral = check OAS' are up to date
        • check-skipped = check OAS' are up to date
      • any of [🛡 GitHub branch protection]:
        • check-success = integration-tests
        • check-neutral = integration-tests
        • check-skipped = integration-tests
      • any of [🛡 GitHub branch protection]:
        • check-success = build-test-images (frontend) / build-test-images
        • check-neutral = build-test-images (frontend) / build-test-images
        • check-skipped = build-test-images (frontend) / build-test-images
      • any of [🛡 GitHub branch protection]:
        • check-success = SonarCloud Code Analysis
        • check-neutral = SonarCloud Code Analysis
        • check-skipped = SonarCloud Code Analysis

@giancarloromeo giancarloromeo added the 🤖-automerge marks PR as ready to be merged for Mergify label Dec 9, 2025
sanderegg
sanderegg approved these changes Dec 9, 2025
View reviewed changes
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

pcrespov
pcrespov approved these changes Dec 9, 2025
View reviewed changes
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 9, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@giancarloromeo giancarloromeo merged commit 1aae10c into ITISFoundation:master Dec 9, 2025
143 of 146 checks passed
@giancarloromeo giancarloromeo deleted the fix8680/improve-stalled-tasks-check branch December 9, 2025 16:29
@GitHK GitHK mentioned this pull request Dec 12, 2025
Closed
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@GitHK GitHK GitHK approved these changes

@pcrespov pcrespov pcrespov approved these changes

@odeimaiz odeimaiz odeimaiz approved these changes

@sanderegg sanderegg sanderegg approved these changes

@matusdrobuliak66 matusdrobuliak66 Awaiting requested review from matusdrobuliak66

@bisgaard-itis bisgaard-itis Awaiting requested review from bisgaard-itis

Assignees

@giancarloromeo giancarloromeo

Labels

🤖-automerge marks PR as ready to be merged for Mergify a:celery-library a:storage issue related to storage service

Projects

None yet

Milestone

Delirium Noel

Development

Successfully merging this pull request may close these issues.

[Bug]: Stalled tasks timeout is too low

5 participants

@giancarloromeo @GitHK @pcrespov @odeimaiz @sanderegg