[tx] Lazy inference engine initialization

[tyler-griggs]

Summary

• Defer vLLM inference engine creation from create_model() to first sampling-related call (save_sampler_checkpoint or sample)
• SFT scripts that never sample no longer spin up inference engines
• RL scripts lazily initialize engines on first save_weights_and_get_sampling_client() call

---

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 5 additional findings.

[gemini-code-assist]

Code Review

This pull request refactors the initialization of inference engines to be lazy, deferring their creation until they are first needed for sampling. This is a good optimization that avoids allocating resources for inference engines in training-only workflows and introduces a mechanism to sleep inference engines during training passes to free up GPU memory. A medium-severity Denial of Service vulnerability was identified, as the sample method does not validate user-provided sampling parameters, which could be abused to cause excessive resource consumption. Additionally, a critical issue exists where the new lazy initialization logic can lead to a crash if num_inference_engines is configured to 0, requiring graceful handling in sample() and save_sampler_checkpoint().

[gemini-code-assist]

While lazy initialization is a great improvement, this change introduces a regression where sample() will crash if num_inference_engines is 0. Previously, there was a check (though likely buggy) to handle this. With lazy init, _ensure_inference_engines() will create an InferenceEngineClient with 0 engines, which then causes a crash inside _inference_engine_client.sample().

Please add a check to ensure there are engines available before proceeding with sampling.

        if not self._inference_engine_client or not self._inference_engine_client.engines:
            error = types.ErrorResponse(
                error="Sampling not enabled. Inference engines were not initialized (num_inference_engines=0 in SkyRL config).",
                status="error",
            )
            return {req_id: error for req_id, _, _, _, _ in prepared_batch.request_batch_slices}

[gemini-code-assist]

Similar to the sample method, this will crash if num_inference_engines is 0 because _dispatch.save_weights_for_sampler() will fail when there are no engines in the InferenceEngineClient. The weight sync should be skipped if no inference engines are configured.

        if self._inference_engine_client and self._inference_engine_client.engines:
            asyncio.run(self._dispatch.save_weights_for_sampler())
            logger.info(f"Synced weights for {model_id} to inference engines via NCCL")
        else:
            logger.info("Skipping sampler weight sync: no inference engines configured.")