You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently GAME's scoring module is tightly coupled with evaluation, and the output schema of the computed scores are not user friendly, because the computed scores can't be joined back to the original data for downstream use cases.
We would like to have an improved GAME scoring module with the following features:
Evaluation will no longer be part of the scoring module, as we shouldn't assume the label/response is available for evaluation purpose for each input example to be scored.
Input data schema: compared to the Avro schema for training data, the scoring data's schema is almost identical except the following three things:
1 Each input example is expected to contain a String typed field called uid (for Avro record), which should represent the unique id of each input example to be scored
2 The response and weight field is no longer required as part of the input
3 If the offset is included as part of the input, then the final score would be offset + scoreComputedFromModel
Output score schema: will leverage ScoringResultAvro.avsc found in photon-avro-schemas.
The text was updated successfully, but these errors were encountered:
Currently GAME's scoring module is tightly coupled with evaluation, and the output schema of the computed scores are not user friendly, because the computed scores can't be joined back to the original data for downstream use cases.
We would like to have an improved GAME scoring module with the following features:
1 Each input example is expected to contain a
String
typed field calleduid
(for Avro record), which should represent the unique id of each input example to be scored2 The response and weight field is no longer required as part of the input
3 If the offset is included as part of the input, then the final score would be
offset + scoreComputedFromModel
ScoringResultAvro.avsc
found inphoton-avro-schemas
.The text was updated successfully, but these errors were encountered: