Is there documentation for configuring modelsetting.json?

[lianxmfor]

Is there documentation for configuring modelsetting.json? I would like to configure the metadata of the model, input/output tensor, etc.

[adriangonz]

Hey @lianxmfor, good catch!

Unfortunately, we don't have any docs yet for the different settings (both at the server and model levels). We are planning to add a section documenting this on the short-term as part of MLServer's new docs site.

In the meantime, although not ideal, it should be relatively straightforward to see the different fields directly in the ModelSettings class of the mlserver/settings.py file.

MLServer/mlserver/settings.py

Lines 49 to 66 in 4946322

	class ModelSettings(BaseSettings):
	class Config:
	env_prefix = ENV_PREFIX_MODEL_SETTINGS
	name: str = ""
	# Model metadata
	platform: str = ""
	versions: Optional[List[str]] = []
	inputs: Optional[List[MetadataTensor]] = []
	outputs: Optional[List[MetadataTensor]] = []
	# Custom model class implementation
	implementation: PyObject = "mlserver.model.MLModel" # type: ignore
	# Model parameters are meant to be set directly by the MLServer runtime.
	# However, it's also possible to override them manually.
	parameters: Optional[ModelParameters] = None

You can also check some of the internal examples, like the one used for our tests: https://github.com/SeldonIO/MLServer/blob/e34c4b9fe2f646e3f890bd49b0dbfb8c8ba36dab/tests/testdata/model-settings.json

I'll keep this issue opened as part of the ongoing documentation efforts.

[lianxmfor]

Thank you for providing so much information!