Multiple observations from simulator, difference between sbi package and sbibm

[gsujan]

Hi

As far i can tell, this package is built using the sbi package link
The sbi library currently does not seem to support multiple observations, i.e the simulator output should have batch size of 1. So generating time series data shouldn't be possible.

This is enforced in function check_for_possibly_batched_x_shape in
user_input_checks

In sbibm package, you have the example code with number of observations as an argument.
observation = task.get_observation(num_observation=1) # 10 per task

According to the sbi package, this shouldn't be possible. Did you use some workaround or am i misinterpreting something ?

[jan-matthis]

Hi @gsujan,

the benchmark is performed for one observation at a time -- to get an estimate about the variance in performance for different observations and across repeats.

Best, Jan-Matthis

[gsujan]

Thanks @jan-matthis for you quick reply.

Just out of curiosity, do you have an explanation on why single observation limitation exists in sbi. I am working on a problem where we want to use multiple real world observations to get a refined posterior. The multi round inference tutorial allows the posterior to be refined but only for single observation. Is there an variation of multi round inference where different observations can be used.

[jan-matthis]

Sure!

It is certainly possible to extend sbi to a multiple observation setting. Is your data i.i.d.? If so, you could for example easily adopt the code for NRE or NLE -- you would just need to change the potential function used during MCMC sampling. This is for example discussed in the paper by Hermans et al. (2020) in section 5.3.2. Or alternatively, one could use the implementation of (S)NPE with exchangeable neural networks (see, e.g., Chan et al. 2018). I will close this issue but definitely feel free to open a new one over in the sbi repo to continue discussing. All of the above would be great extensions and I'd be glad to help out with it.