Document how to implement custom models

[HaoLiu56]

Issue description

Provide a short description.

I wonder if we could implement a custom model instead of GP class model.
Specifically, suppose we build a random forest model, and it also have mean and variance for data points. Is it possible to do Bayesian Optimization with this model under BoTorch framework?

Code example

For example:
model = CustomModel(train_X=init_x, train_Y=init_y )

System Info

Please provide information about your setup, including

• BoTorch Version (0.10.0)
• GPyTorch Version (1.11)
• PyTorch Version (2.2.0.post100)
• Computer OS: mac OS Sonoma

[Balandat]

Yes, this is generally possible. BoTorch's Model class only requires you to define a posterior() method that returns a Posterior object, the only requirement of which is to implement an rsample() function for drawing posterior samples (the EnsemblePosterior class provides an implementation for that). Then the MC acquisition function machinery will work with such a setup (EnsemblePosterior also implements mean and variance properties, so some other analytic acquisition functions will also work).

That said, note that in order to use gradient-based optimization of the acquisition function (via the standard optimize_acqf() method) you will need to have the samples drawn from the posterior be differentiable w.r.t. to the input to the posterior() method (this is not the case for Random Forest models). In that case you'll have to perform the acquisition function optimization with gradient-free methods (e.g. CMA-ES or similar methods).

[HaoLiu56]

Great! Could you include some tutorials notebook for this? I'm fairly new to BoTorch.