Adversarial data collection with Gradio

[Ram81]

Project Title: Adversarial data collection with Gradio

Description:

This project will focus on building an infrastructure to allow exposing models submitted to EvalAI as demos in order to collect adversarial data for the model. As part of the project, we will integrate Gradio with our code upload challenge pipeline to allow deploying the models as web services. Additionally, this web service will record all interactions to curate a "in-the-wild" dataset for each submission.

Deliverables:

• Build system design for EvalAI and Gradio integration
• Set up Gradio integration for a single VQA demo as a proof of concept
  • Write a gradio interface wrapper that is modular for model inputs
  • Integrate the gradio interface with dockerized model for inference
• Set up auto launching of a worker service for deploying a Gradio app using a celery task from EvalAI django backend
• Add frontend controls to deploy a Gradio web service for a single submission made by participant to a code upload challenge.
• Add support to log interactions made by a user on the gradio web service and push the interactions to a database table specific to a demo.
• Add API support to export the logged interactions for a demo
• Add frontend changes to allow users to download the logged interactions in a standard dataset format

Mentors: - Ram Ramrakhya, Rishabh Jain

Skills Required: - Python, Django, AngularJS, AWS

Project size - 175 hours

Difficulty - Hard

Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)

Important Links:

• EvalAI Website: eval.ai
• EvalAI Github repository: Cloud-CV/EvalAI
• EvalAI Docs: http://evalai.readthedocs.io/en/latest
• GSoC Proposal Template: Cloud-CV/GSoC-Ideas/wiki/GSOC-2020-Proposal-Template
• Gitter Channel: gitter.im/Cloud-CV
• Mailing list: groups.google.com/forum/#!forum/cloudcv