evalRS-KDD-2023

Official Repository for EvalRS @ KDD 2023: a Rounded Evaluation of Recommender Systems

Overview

EvalRS 2023 will be held at KDD 2023, August 7th (Long Beach, CA).

Following the success of the first edition, the workshop is back with the same focus on rounded evaluation of recommender systems (fairness, robustness and trustworthyness), but with a new, expanded format.

For the first time, the event will mix a traditional paper track and an open-source hackaton, inspiring participants to "live and breath" the evaluation challenges with working code. Following our tradition, we will run the entire workshop in the open, and give back to the community all the artifacts produced for the event.

Thanks to our sponsors, we will award monetary prizes for innovative papers, student contributions and best projects.

Check back soon this repo and the official website for keynote speakers and final logistics details: if you have questions about paper submission or partecipation during KDD, send us a note at fede a_t stanford d_o_t edu or jacopo d_o_t tagliabue a_t nyu d_o_t edu.

Quick links:

• The workshop website (agenda, speakers, logistics)
• The paper submission link (to submit your paper)
• The workshop paper (scholarly presentation, background, literature)
• RecList OS library and paper (it will be used for the event)
• EvalRS 2022 - data challenge, paper, keynotes, after-math

Organizers

EvalRS 2023 is brought to you with love by:

• Federico Bianchi - Stanford
• Patrick John Chia - Coveo
• Ciro Greco - Bauplan
• Gabriel Moreira - NVIDIA
• Claudio Pomo - Politecnico di Bari
• Davide Eynard - Mozilla AI
• Fahd Husain - Mozilla AI
• Jacopo Tagliabue - NYU, Bauplan

Sponsors

The event is supported by Mozilla AI. Exciting news coming soon!

Citations

When citing our work, please cite both the RecList original paper and the workshop presentation:

@inproceedings{10.1145/3487553.3524215,
    author = {Chia, Patrick John and Tagliabue, Jacopo and Bianchi, Federico and He, Chloe and Ko, Brian},
    title = {Beyond NDCG: Behavioral Testing of Recommender Systems with RecList},
    year = {2022},
    isbn = {9781450391306},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3487553.3524215},
    doi = {10.1145/3487553.3524215},
    pages = {99–104},
    numpages = {6},
    keywords = {recommender systems, open source, behavioral testing},
    location = {Virtual Event, Lyon, France},
    series = {WWW '22 Companion}
}

@inproceedings{Bianchi2023EvalRS2W,
  title={EvalRS 2023. Well-Rounded Recommender Systems For Real-World Deployments},
  author={Federico Bianchi and Patrick John Chia and Ciro Greco and Claudio Pomo and Gabriel de Souza Pereira Moreira and Davide Eynard and Fahd Husain and Jacopo Tagliabue},
  year={2023}
}

License

All the code is released "as is" under an open MIT license.