Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eval/vbench #312

Merged
merged 18 commits into from
May 13, 2024
Merged

Eval/vbench #312

merged 18 commits into from
May 13, 2024

Conversation

BeachWang
Copy link
Collaborator

将pypi的Vbench应用到sandbox中。目前pypi上的Vbench版本存在几个问题:

  1. 由于缺少第三方库代码,motion_smoothness无法测评;
  2. Vbench用decord加载视频,在同时测评多个dimension指标时,会出现返回类型帧的类型不一致问题,导致转numpy出错。

@BeachWang BeachWang added documentation Improvements or additions to documentation enhancement New feature or request dj:multimodal issues/PRs about multimodal data processing labels May 8, 2024
@BeachWang BeachWang added this to the DJ-SORA milestone May 8, 2024
@BeachWang BeachWang self-assigned this May 8, 2024
README_ZH.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@HYLcool HYLcool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BeachWang BeachWang merged commit 0d81116 into main May 13, 2024
4 checks passed
@BeachWang BeachWang deleted the eval/vbench branch May 13, 2024 12:17
@@ -1,3 +1,8 @@
torch>=1.11.0,<2.0.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里 <2.0.0 是 vbench 强制要求的吗?比赛中提供的 docker 是 torch 2.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dj:multimodal issues/PRs about multimodal data processing documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants