make ModelResponse objects deterministic

[comhar]

fixes #14

LiteLLM ModelResponse objects have id and created_at fields that are generated dynamically. Even when we use cachy to cache the LLM response these dynamic fields create diffs which makes code review more challenging.

This pr patches ModelResponseBase so that id and created_at always have fixed values.

@patch
def __init__(self: ModelResponseBase, id=None, created=None, *args, **kwargs): 
    self._orig___init__(id='chatcmpl-xxx', created=1000000000, *args, **kwargs)

@patch
def __setattr__(self: ModelResponseBase, name, value):
    if name == 'id': value = 'chatcmpl-xxx'
    elif name == 'created': value = 1000000000
    self._orig___setattr__(name, value)

[comhar]

ModelResponse fields like id and created_at are included in the LLM prompt. As this pr changes the value of those fields that's why we're seeing new items being added to cachy. If we end-up merging this pr then we should create a follow-up pr that resets the cache and removes the stale cache items. The easiest way to do this would be to delete cachy.jsonl and just re-run the notebook.

[comhar]

@RensDimmendaal here's an alternative approach to making LiteLLM ModelResponse objects deterministic. wdyt? Would you mind trying it out on your machine and confirming that no diff is generated on your end?

[comhar]

@RensDimmendaal here's an alternative approach to making LiteLLM ModelResponse objects deterministic. wdyt? Would you mind trying it out on your machine and confirming that no diff is generated on your end?

Actually hold off on the review for now, looks like litellm.types.utils.time.time is not as isolated as I first thought 😞

[comhar]

@RensDimmendaal I think I found a clean enough solution now that seems to do the job.

@patch
def __init__(self: ModelResponseBase, id=None, created=None, *args, **kwargs): 
    self._orig___init__(id='chatcmpl-xxx', created=1000000000, *args, **kwargs)

@patch
def __setattr__(self: ModelResponseBase, name, value):
    if name == 'id': value = 'chatcmpl-xxx'
    elif name == 'created': value = 1000000000
    self._orig___setattr__(name, value)

[jph00]

Well spotted! :)