Added retries parameters to pipeline.draw()

[nickprock]

Related Issues

• fixes Add retries parameter to pipeline.draw() to help mitigate mermaid server issues #9045

Proposed Changes:

• Implemented an exponential retry mechanism
• Added max_retries and initial_delay parameters to control retry behavior and allow more customization
• Included unit test to check the correct operation of the retry mechanism and exponential backoff

How did you test it?

unit tests

Notes for the reviewer

@anakin87 check if it's ok, you are cited in the issue.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[coveralls]

Pull Request Test Coverage Report for Build 14076804129

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 8 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.005%) to 90.154%

---

Files with Coverage Reduction	New Missed Lines	%
core/pipeline/draw.py	8	87.9%

Totals
Change from base Build 14060960165:	0.005%
Covered Lines:	9898
Relevant Lines:	10979

---

💛 - Coveralls

[davidsbatista]

This is good, and I agree with the PR, but in the future, we probably should:

• only run this test if this file is changed in a PR and not for every single PR
• make sure that, regardless of changes, it runs at least 1 or 2 a week

My point is that running it for every PR might be a might too much charge for mermaid servers - alternatively, we can also host a Docker with a mermaid server and run against that server

[julian-risch]

@davidsbatista @Amnah199 We could consider using request_with_retry from utils:

haystack/haystack/utils/requests_utils.py

Line 45 in dae8c7b

res = request_with_retry(

[davidsbatista]

I had another look at the PR; I overlooked the fact that retry is being added to the code that actually calls the endpoint and draws the image. Initially, I thought this was only being added to the test.

IMO this retry mechanism should only be added to test and not the code of the component.

[Amnah199]

@nickprock Thanks a lot for this great contribution — we really appreciate your continued support and engagement with Haystack! 🙌

We had an internal discussion around this issue and, at the moment, we're unsure about the best way to proceed. Introducing retries in pipeline.draw() could unintentionally increase traffic to external services like mermaid.ink, which may not be ideal.
For now, we’ve decided to put this issue on hold while we explore alternative solutions, such as potentially hosting our own instance of mermaid. We'll keep you updated as we make progress!

[davidsbatista]

@nickprock here is follow up issue: #9107

[davidsbatista]

Hi @nickprock, thank you again for your initiative; unfortunately, two things happened. Although not explicitly stated in #9045, the idea was to add the retry to tests only, not to the code itself.

Plus, we are now skipping those tests since they tend to fail and block daily PRs. As pointed out, there's another issue, #9107, to tackle this.

I want to thank you once again for your initiative and contribution and feel free to suggest any suggestions on the new issue I shared.

[nickprock]

Thanks @davidsbatista @Amnah199 🙏