Suggestion: Make dependency of metrics on distill functions explicit?

[neubig]

Within Zeno, some metrics are dependent on distill functions. For instance, in this example the avg_bert_score metric is dependent on the bert_score distill function.

However, it is possible that through human error, one might add a metric to ZenoParameters but forget to add the corresponding distill function, leading to a hard-to-debug error.

As a suggestion, what about adding an optional argument to specify dependencies as follows:

@distill
def my_distill(df: DataFrame, ops: ZenoOptions) -> DistillReturn:
  ...

@metric(dependencies={"my_distill": my_distill})
def my_metric(df: DataFrame, ops: ZenoOptions) -> MetricReturn:
  ...

The key indicates the destination column of the distill function, and the value specifies the function itself. Then, the metric decorator could add an automatic check that the dependencies have been calculated before-hand and throw a comprehensible error if not.