SMC marginal test failing due to input type

This SMC  test is failing in main https://github.com/pymc-devs/pymc/runs/4740953565?check_suite_focus=true#step:7:746


This is somewhat related to #5263, we wouldn't need those nan otherwise.