Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ASoC 2022] DeepRec supports multiple evaluator. #230

Closed
liutongxuan opened this issue May 25, 2022 · 0 comments
Closed

[ASoC 2022] DeepRec supports multiple evaluator. #230

liutongxuan opened this issue May 25, 2022 · 0 comments

Comments

@liutongxuan
Copy link
Member

liutongxuan commented May 25, 2022

Background

This is an advance subject of ASoC 2022 and #231 .

At present, DeepRec cannot support the evaluation of very large models (single node cannot be loaded), multiple PS are required to load large models, and multiple workers are used for distributed evaluation.

Target

  1. Design and implement the capability large model evaluation, support multiple PS loading large model.
  2. Design and implement multiple evaluator node in one job.

Difficulty

Advance

Mentor

@candyzone candy.dc@alibaba-inc.com

Output Requirements

Proficiency in C++ and Python;
Get to know DeepRec;
Able to complete the development under the guidance of the mentor;
Have a certain understanding and interest in deep learning recommendation engines;

背景

这是一个阿里巴巴编程之夏 2022 的基础课题 #231 .

DeepRec 支持多evaluator评估:目前DeepRec下无法支持超大模型(单节点无法加载)的评估,需要多个ps加载大模型,并且使用多worker进行分布式评估。

目标

1)支持超大模型通过多PS方式加载模型,实现Evaluation.
2)支持一个任务中使用多个Evaluator节点进行评估。

难度

进阶

导师

@candyzone candy.dc@alibaba-inc.com

产出要求

熟练掌握C++和Python;
能够在导师的指导下熟悉并理解相关的代码
了解 DeepRec;
对深度学习推荐引擎有一定了解和兴趣;

@liutongxuan liutongxuan changed the title [ASoC 2022] DeepRec supports multiple evaluator (DeepRec 支持多evaluator评估) [ASoC 2022] DeepRec supports multiple evaluator. May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant