Skip to content

Change SamplingStrategy interface #160

@nrfulton

Description

@nrfulton

The SamplingStrategy interface has a major issue: everything about a sampling strategy is (unnecessarily and sneakily) stateful because the generate and validate fields carry around a MelleaSession. We should refactor this code before accumulating too much code that depends on the current interface. The strategy itself should be stateless, and sessions should be passed into the sample method explicitly (as the first parameter).

import mellea
from mellea.stdlib.sampling import RejectionSamplingStrategy

m1 = mellea.start_session(model_id="model_foo")

def generate(*args, **kwargs):
    m1.backend._generate_from_raw(...)

strategy = RejectionSamplingStrategy(..., generate=generate)

m2 = mellea.start_session(model_id="model_bar") 
m2.instruct("blah", strategy=strategy)

Having the generate method close over the context is ruinous for a lot of reasons. Here are the two biggest issues:

  1. This code will not result in any calls to m2._backend.generate_*; only m1._backend.generate_from_context gets called. This is very surprising, because we called m2.instruct.
  2. If you make async calls, then any code doing sampling will almost always have race conditions.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions