Make CodeEval respect device_eval_batch_size (#2969)

* Make CodeEval respect device_eval_batch_size

* fix

* fix

* Avoid materializing dataset in memory

* fix

* fix

* Helper func

* fix

* Remove todo

* fix documentation

* use pre-commit run

* Tests

* fix error msg

* left padding

* final touches

* Fix gpu test

* Tests

* Return tensor

* fix

* pyright

* fix pyright

* Allow subset batches

* pyright

---------

Co-authored-by: Eitan Turok <eitan.turok@databricks.com>
Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>