Do not force download by default in ORTModel

[fxmarty]

We passed by default force_download=True, although the caching works well when passing force_download=False. Especially when passing from_transformers=False, everything works as expected as the onnx model is loaded from the cache hub/ subfolder.

When passing from_transformers=True, although we cache only the PyTorch model (before the new transformers cache system in transformers/ cache subfolder, in hub/ subfolder in the next transformers release), the force_download argument is anyway ignored in _from_transformers so this PR has no impact there:

optimum/optimum/onnxruntime/modeling_ort.py

Lines 239 to 303 in e903324

	@classmethod
	def _from_transformers(
	cls,
	model_id: str,
	save_dir: Union[str, Path] = default_cache_path,
	use_auth_token: Optional[Union[bool, str, None]] = None,
	revision: Optional[Union[str, None]] = None,
	force_download: bool = False,
	cache_dir: Optional[str] = None,
	**kwargs,
	):
	"""
	Converts a vanilla Transformers model into an optimized model using `transformers.onnx.export_onnx`.
	Arguments:
	model_id (`str` or `Path`):
	Directory from which to load
	save_dir (`str` or `Path`):
	Directory where the onnx model should be saved, default to `transformers.file_utils.default_cache_path`, which is the cache dir for
	transformers.
	use_auth_token (`str` or `bool`):
	Is needed to load models from a private repository
	revision (`str`):
	Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id
	cache_dir (`Union[str, Path]`, *optional*):
	Path to a directory in which a downloaded pretrained model configuration should be cached if the
	standard cache should not be used.
	force_download (`bool`, *optional*, defaults to `False`):
	Whether or not to force the (re-)download of the model weights and configuration files, overriding the
	cached versions if they exist.
	kwargs (`Dict`, *optional*):
	kwargs will be passed to the model during initialization
	"""
	# create local save dir in cache dir
	save_dir = Path(save_dir).joinpath(model_id)
	save_dir.mkdir(parents=True, exist_ok=True)
	kwargs["model_save_dir"] = save_dir
	# reads pipeline task from ORTModelForXXX class if available else tries to extract from hub
	if cls.export_feature is not None:
	task = cls.export_feature
	else:
	task = HfApi().model_info(model_id, revision=revision).pipeline_tag
	if task in ["sentiment-analysis", "text-classification", "zero-shot-classification"]:
	task = "sequence-classification"
	elif task in ["feature-extraction", "fill-mask"]:
	task = "default"
	# 2. convert to temp dir
	# FIXME: transformers.onnx conversion doesn't support private models
	preprocessor = get_preprocessor(model_id)
	model = FeaturesManager.get_model_from_feature(task, model_id)
	_, model_onnx_config = FeaturesManager.check_supported_model_or_raise(model, feature=task)
	onnx_config = model_onnx_config(model.config)
	# export model
	export(
	preprocessor=preprocessor,
	model=model,
	config=onnx_config,
	opset=onnx_config.default_onnx_opset,
	output=save_dir.joinpath(ONNX_WEIGHTS_NAME),
	)
	kwargs["config"] = model.config.__dict__
	# 3. load normal model
	return cls._from_pretrained(save_dir.as_posix(), **kwargs)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[regisss]

LGTM @fxmarty!

If the force_download argument is not used in _from_transformers, maybe we should remove it what do you think?

[philschmid]

That's awesome! But we have to add test to validate that it is working as expected. Especially since we are storing the "model.onnx" files at the top level with out any hash. So there is no way for the library to know if the cached "model.onnx" is the correct one or not.

We can add the local_files_only argument to the from_pretrained method and have then working and failing tests for "cached" and regular models. As well as for loading different models after each other to make sure we have the correct ones loaded.
See documentation: https://huggingface.co/docs/huggingface_hub/package_reference/file_download#huggingface_hub.hf_hub_download.local_files_only

We should also check that this works for the seq2seq models.

And we should check if the new cache system allows better distribution of onnx files. Currently, when a user loads and saves a onnx file it is saved at the top cache directory with model.onnx, which means if he loads a new model "from" cache it uses the old model. Thats why i went with "force_download" in the beginning.

[fxmarty]

When loading an onnx model with from_transformers=False, here's where it is loaded from in the hub/ subfolder:

For seq2seq models (with from_transformers=False), we have:

which does work as well. Thanks for the suggestion to add tests, I will try to add some!

When from_transformers=True, the argument force_download is ignored.

In case we passed from_transformers=True, in the old caching system, we indeed get e.g. a transformers/facebook/levit-256/model.onnx file with no hash recorded. However, even if this file exists, it is NOT loaded into the InferenceSession and the cached pytorch model is converted to onnx and loaded into an InferenceSession (the file is overwritten at each load).

Under the new caching system, if we pass from_transformers=True, we get e.g. a hub/facebook/levit-256/model.onnx with no hash recorded as well. Looking at the timestamps, this file is overwritten at every conversion.

In any case, I think the case from_transformers=True is independent from this PR.

[philschmid]

LGTM!