Skip to content

fix: stress test setup process and report structure workflow for Dify 1.15.0+#38194

Merged
crazywoola merged 1 commit into
langgenius:mainfrom
Blackoutta:fix/stress_test_scripts
Jun 30, 2026
Merged

fix: stress test setup process and report structure workflow for Dify 1.15.0+#38194
crazywoola merged 1 commit into
langgenius:mainfrom
Blackoutta:fix/stress_test_scripts

Conversation

@Blackoutta

@Blackoutta Blackoutta commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR updates the stress-test setup and SSE benchmark scripts so they can run smoothly on Dify 1.15.0+ source deployments.

It fixes the setup flow around current console authentication, initialized-instance handling, plugin installation responses, workflow import warnings, setup step failure propagation, Locust dependency conflicts, and per-run report output.

Fixes #38193.

Changes

  • Encode console login passwords like the web client and persist the auth/CSRF cookies returned by login.
  • Send console auth headers and cookies from shared config helpers.
  • Support both brand-new Dify instances and already-initialized instances with admin credentials from environment variables.
  • Make setup scripts return non-zero on real failures so setup_all.py stops at the root cause.
  • Treat already-installed/no-task plugin install responses as non-blocking when no error code is present.
  • Accept workflow imports that complete with warnings when an app_id is returned.
  • Run Locust through uvx --from locust locust to avoid API dependency conflicts.
  • Group each benchmark run's reports under a timestamped directory and print the custom SSE metrics JSON path.
  • Add focused tests for the setup script behavior.

Video

fix_stress_test.mp4

Validation

  • uv run --project api pytest scripts/stress-test/test_setup_scripts.py
  • uv run --project api python -m py_compile scripts/stress-test/setup_all.py scripts/stress-test/test_setup_scripts.py
  • bash -n scripts/stress-test/run_locust_stress_test.sh
  • uvx --from locust locust -f scripts/stress-test/sse_benchmark.py --list

@Blackoutta Blackoutta changed the title Fix stress test setup workflow for Dify 1.15.0+ fix: stress test setup workflow for Dify 1.15.0+ Jun 30, 2026
@Blackoutta Blackoutta changed the title fix: stress test setup workflow for Dify 1.15.0+ fix: stress test setup process and report structure workflow for Dify 1.15.0+ Jun 30, 2026
@Blackoutta Blackoutta marked this pull request as ready for review June 30, 2026 06:11
@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 30, 2026
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jun 30, 2026
@crazywoola crazywoola added this pull request to the merge queue Jun 30, 2026
Merged via the queue into langgenius:main with commit b4be4d9 Jun 30, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stress test scripts are hard to run on Dify 1.15.0+

2 participants