Test for valid use of (S)NPE API

[psteinb]

This PR attempts a solution to #23 without (yet) introducing test categories a la @pytest.mark.slow (see https://github.com/mackelab/sbi/blob/86256e02c1080965795e65062c4ab9d3a19015d2/tests/linearGaussian_snpe_test.py#L196)

[psteinb]

btw, when run, this unit tests results in:

        # For torch < 1.8 log_abs_det_jacobian is returned for each dimension.
        if vals.ndim > 1 and vals.shape[1] == dim_parameters:
            vals = vals.sum(-1)
    
>       assert (
            vals.numel == batch_size
        ), "Mismatch in batch size, took sum over whole batch?"
E       AssertionError: Mismatch in batch size, took sum over whole batch?

../../sbibm/utils/torch.py:105: AssertionError

which is related to #15 I guess

[jan-matthis]

which is related to #15 I guess

Yes, this is due to #15. @janfb will fix this beginning of next week, that will iron it out

[jan-matthis]

I'd prefer keeping the tests deterministic, i.e., let's just run them on observation 1 and 2 only if 10 is too slow. How long does this test take on your machine?

[psteinb]

With 2 observations and 2.500 simulated samples per task, all 6 tests in total run for 220s on my box. That is still quite long and mostly due to NPE running epochs. In this case, this is a hen-and-egg problem: we balance simulation budget versus expressiveness of the predictor.
I'd propose to ditch the SLCP task or at least one gaussian task. Limiting the number of epochs would be most ideal - but impossible I guess at the moment.

[psteinb]

I boiled it down to 2 tasks and about 140s to complete (9e91818). Still quite long. I guess the central question is, what we want to achieve with this test. If it is really only about the interface, then one task with one observation is enough.

[jan-matthis]

Yes, I think we only want to do interface testing as part of sbibm. I have just merged a PR (#33) that adds an argument to be able to set max_num_epochs explicitly. How about running the test for a low maximum number of epochs and only checking that the shape of the return is correct?

[psteinb]

Cool, these tests are now going at 30s on my box. That is still long, but at least c2st appears to return something meaningful. e642d77

[jan-matthis]

Cheers!