test: add test for hooks #38
Description
We are expecting that hooks to create a certain log (JSONL).
So we can check the generated JSONL to ensure that hooks are logging correctly.
- Read run.jsonl
- Validate structure of each Json line (in the future with pydantic models)
- Play the run:
- write on a tmpfile.jsonl
- compare run.jsol each step with the step from the gameplay
- Compare the run.jsonl and tmpfile.jsonl
This test for hook (now in test_runs.py) could be a really powerful way of testing. You can play balatro like a normal player, and the hook records all the runs: every action (aka function calls) and every game state. If we have implemented ALL aspects of the bot framework correctly, we should have perfect reproducibility by simply replaying function calls.
Maybe this can be abuse by iterating LLM feature devs against a test suite of runs generated by various bots (dummy or smart). Of course, if we want to scale testing (and run.jsonl validation is super important IMHO), we need to optimize the game rendering (ideally headless mode).