Two retry loops still hand-rolled: URLFetchUtil retries without delay, PythonProxyClient waits a constant delay

[aglinxinyuan]

Task Summary

Two retry loops were left hand-rolled when #7119 consolidated the rest onto RetryUtil.withBackoff, because converting them changes runtime behavior rather than just structure. Each needs a decision before it can move.

1. URLFetchUtil.getInputStreamFromURL (common/workflow-operator)

def getInputStreamFromURL(urlObj: URL, retries: Int = 5): Option[InputStream]

• Retries 5x with no delay at all, and returns Option rather than throwing.
• Adopting the util's backoff means a dead URL costs ~3 s (200+400+800+1600) before giving up, where today it fails immediately. That is arguably the fix — an unthrottled 5x retry against a remote host is a small retry storm — but it changes how long a URL-scan operator takes to fail.
• The Option return is a one-line adapter (Try(...).toOption), so the delay is the only real question.

2. PythonProxyClient's Flight-connect loop (amber)

while (!connected && tryCount <= MAX_TRY_COUNT) { ... Thread.sleep(UNIT_WAIT_TIME_MS) ... }

• Waits a constant UNIT_WAIT_TIME_MS between attempts, not a doubling one, so the util needs a delay-multiplier knob (1 = constant) before it can host this.
• Closes the Flight client between attempts; that per-attempt cleanup has to move into the operation body.
• Throws WorkflowRuntimeException on give-up, where the util wraps in RuntimeException. Either the util grows a give-up hook, or this call site accepts the different type — worth deciding deliberately, since worker-bring-up failures are surfaced to users.

Both are candidates, not obligations: if the answer for either is "the current behavior is what we want", closing this with that note recorded is a fine outcome.

Task Type

• Refactor / Cleanup