Skip to content

support using inference_base_image in transformers processor #3781

@happywhaler

Description

@happywhaler

metadata["azureml.base_image"] = "mcr.microsoft.com/azureml/curated/foundation-model-inference:" \

        if self._inference_base_image:
            metadata["azureml.base_image"] = self._inference_base_image
        elif self._vllm_enabled:
            mlclient = get_mlclient("azureml")
            vllm_image = mlclient.environments.get("foundation-model-inference", label="latest")
            metadata["azureml.base_image"] = "mcr.microsoft.com/azureml/curated/foundation-model-inference:" + str(
                vllm_image.version
            )
            logger.info("Metadata: {}".format(metadata))

we don't always want to take the latest vllm image, would be awesome to be able to use this argument here as well as latest is not always working (alternatively, allow setting a different label).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions