Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GAME scoring module improvement #71

Closed
XianXing opened this issue May 12, 2016 · 2 comments
Closed

GAME scoring module improvement #71

XianXing opened this issue May 12, 2016 · 2 comments

Comments

@XianXing
Copy link
Contributor

XianXing commented May 12, 2016

Currently GAME's scoring module is tightly coupled with evaluation, and the output schema of the computed scores are not user friendly, because the computed scores can't be joined back to the original data for downstream use cases.

We would like to have an improved GAME scoring module with the following features:

  • Evaluation will no longer be part of the scoring module, as we shouldn't assume the label/response is available for evaluation purpose for each input example to be scored.
  • Input data schema: compared to the Avro schema for training data, the scoring data's schema is almost identical except the following three things:
    1 Each input example is expected to contain a String typed field called uid (for Avro record), which should represent the unique id of each input example to be scored
    2 The response and weight field is no longer required as part of the input
    3 If the offset is included as part of the input, then the final score would be offset + scoreComputedFromModel
  • Output score schema: will leverage ScoringResultAvro.avsc found in photon-avro-schemas.
@XianXing XianXing changed the title GAME scoring with candidate set improvement GAME scoring module improvement May 12, 2016
@XianXing
Copy link
Contributor Author

Related to #47.

@XianXing
Copy link
Contributor Author

XianXing commented Jun 9, 2016

Reopen this issue, because some of our clients at LinkedIn complained the rigidness of the proposed solution.

Will work on a more flexible and easy-to-use scoring module for GAME.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant