generated from eliahuhorwitz/Academic-project-page-template
-
Notifications
You must be signed in to change notification settings - Fork 1
/
index.html
270 lines (222 loc) · 12.2 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="description" content="DESCRIPTION META TAG">
<meta property="og:title" content="SOCIAL MEDIA TITLE TAG"/>
<meta property="og:description" content="SOCIAL MEDIA DESCRIPTION TAG TAG"/>
<meta property="og:url" content="URL OF THE WEBSITE"/>
<meta property="og:image" content="static/image/your_banner_image.png" />
<meta property="og:image:width" content="1200"/>
<meta property="og:image:height" content="630"/>
<meta name="twitter:title" content="TWITTER BANNER TITLE META TAG">
<meta name="twitter:description" content="TWITTER BANNER DESCRIPTION META TAG">
<meta name="twitter:image" content="static/images/your_twitter_banner_image.png">
<meta name="twitter:card" content="summary_large_image">
<meta name="keywords" content="KEYWORDS SHOULD BE PLACED HERE">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>AnimateZoo</title>
<link rel="icon" type="image/x-icon" href="static/images/favicon.ico">
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="static/css/bulma.min.css">
<link rel="stylesheet" href="static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="static/css/bulma-slider.min.css">
<link rel="stylesheet" href="static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="static/css/index.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script src="https://documentcloud.adobe.com/view-sdk/main.js"></script>
<script defer src="static/js/fontawesome.all.min.js"></script>
<script src="static/js/bulma-carousel.min.js"></script>
<script src="static/js/bulma-slider.min.js"></script>
<script src="static/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-2 publication-title">AnimateZoo: Zero-shot Video Generation of Cross-Species Animation via Subject Alignment</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<a href="FIRST AUTHOR PERSONAL LINK" target="_blank">Yuanfeng Xu</a><sup>1</sup>,</span>
<span class="author-block">
<a href="SECOND AUTHOR PERSONAL LINK" target="_blank">Yuhao Chen</a><sup>1</sup>,</span>
<span class="author-block">
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Zhongzhan Huang</a><sup>1</sup>,</span>
<span class="author-block">
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Zijian He</a><sup>1</sup>,</span>
<span class="author-block">
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Guangrun Wang</a><sup>2</sup>,</span>
<span class="author-block">
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Philip Torr</a><sup>2</sup>,</span>
<span class="author-block">
<a href="THIRD AUTHOR PERSONAL LINK" target="_blank">Liang Lin</a><sup>1</sup></span>
</div>
<div class="is-size-5 publication-authors">
<span class="author-block"><sup>1</sup>Sun Yat-sen University,<sup>2</sup>University of Oxford</span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<span class="link-block">
<a href="https://arxiv.org/abs/2404.04946" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="ai ai-arxiv"></i>
</span>
<span>arXiv</span>
</a>
</span>
<span class="link-block">
<a href="https://github.com/JustinXu0/AnimateZoo" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code</span>
</a>
</span>
<span class="link-block">
<a href="https://arxiv.org/pdf/<ARXIV PAPER ID>.pdf" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-youtube"></i>
</span>
<span>Dataset (soon)</span>
</a>
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<section class="card" style="width:60%; margin-top: 30px; margin-bottom: 30px;">
<div class="container is-max-desktop">
<div class="hero-body">
<img src="static/videos/main_cases_1_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/main_cases_2.gif" alt="MY ALT TEXT"/>
<img src="static/videos/main_cases_3.gif" alt="MY ALT TEXT"/>
<img src="static/videos/main_cases_4.gif" alt="MY ALT TEXT"/>
<h2 class="subtitle has-text-centered">
Faithful and controllable cross-species animation results of <b><i>AnimateZoo</i></b>.
Without any parameter tuning, our model supports seamlessly inheriting actions from diverse animal species while preserving the scene and appearance information consistency.
</h2>
</div>
</div>
</section>
<section class="card" style="width:60%; margin-top: 30px; margin-bottom: 30px;">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-2">Abstract</h2>
<div class="content has-text-justified">
<p>
Recent video editing advancements rely on accurate pose sequences to animate subjects.
However, these efforts are not suitable for cross-species animation due to pose misalignment between species
(for example, the poses of a cat differs greatly from that of a pig due to differences in
body structure). In this paper, we present <b><i>AnimateZoo</i></b>, a zero-shot diffusion-based video generator to address this challenging cross-species animation issue,
aiming to accurately produce animal animations while preserving the background. The key technique used in our AnimateZoo is subject alignment, which includes two steps.
First, we improve appearance feature extraction by integrating a Laplacian detail booster and a prompt-tuning identity extractor. These components are specifically designed to capture essential appearance information,
including identity and fine details. Second, we align shape features and address conflicts from differing subjects by introducing a scale-information remover. This ensures accurate cross-species animation.
Moreover, we introduce two high-quality animal video datasets featuring a wide variety of species. Trained on these extensive datasets, our model is capable of generating videos characterized by accurate movements, consistent appearance, and high-fidelity frames, without the need for the pre-inference fine-tuning that prior arts required.
Extensive experiments showcase the outstanding performance of our method in cross-species action following tasks, demonstrating exceptional shape adaptation capability.
</p>
</div>
</div>
</div>
</div>
</section>
<section class="card" style="width:60%; margin-top: 30px; margin-bottom: 30px;">
<div class="hero-body">
<div class="container">
<div class="columns is-centered has-text-centered">
<h2 class="title is-2">Method Overview</h2>
</div>
<img src="static/images/overview.svg" alt="">
</div>
</div>
</section>
<section class="card" style="width:60%; margin-top: 30px; margin-bottom: 30px;">
<div class="container is-max-desktop">
<div class="hero-body">
<h2 class="title is-3">Video Presentation</h2>
<img src="static/videos/more_cases_1_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/more_cases_3_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/more_cases_4_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/more_cases_2_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/more_cases_5_title.gif" alt="MY ALT TEXT"/>
<h2 class="subtitle has-text-centered">
Replacing video subjects with animals exhibiting different appearance characteristics
</h2>
</div>
</div>
</section>
<section class="card" style="width:60%; margin-top: 30px; margin-bottom: 30px;">
<div class="hero-body">
<h2 class="title is-3">Another Application: Creating Something Out of Nothing</h2>
<img src="static/videos/same_backgrd1_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_backgrd4_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_backgrd5_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_backgrd6_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_backgrd7_title.gif" alt="MY ALT TEXT"/>
<h2 class="subtitle has-text-centered">
Generating different animals and actions in the same scenario
</h2>
</div>
<div class="hero-body">
<img src="static/videos/same_subject_desert_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_subject_wild1_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_beach_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/same_subject_wild2_title.gif" alt="MY ALT TEXT"/>
<h2 class="subtitle has-text-centered">
Transplanting the same subject and action across various scenarios
</h2>
</div>
</section>
<section class="card" style="width:60%; margin-top: 30px; margin-bottom: 30px;">
<div class="container is-max-desktop">
<div class="hero-body">
<h2 class="title is-3">Comparison with Advanced Methods</h2>
<img src="static/videos/user_study1_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/user_study2_title.gif" alt="MY ALT TEXT"/>
<img src="static/videos/user_study3_title.gif" alt="MY ALT TEXT"/>
<h2 class="subtitle has-text-centered">
Our method surpasses others in background and subject fidelity, video quality, and coherence
</h2>
</div>
</div>
</section>
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code>@article{xu2024animatezoo,
title={AnimateZoo: Zero-shot Video Generation of Cross-Species Animation via Subject Alignment},
author={Xu, Yuanfeng and Chen, Yuhao and Huang, Zhongzhan and He, Zijian and Wang, Guangrun and Torr, Philip and Lin, Liang},
journal={arXiv preprint arXiv:2404.04946},
year={2024}
}
</code></pre>
</div>
</section>
<footer class="footer">
<div class="container">
<div class="columns is-centered">
<div class="column is-8">
<div class="content">
<p>
This page was built using the <a href="https://github.com/eliahuhorwitz/Academic-project-page-template" target="_blank">Academic Project Page Template</a> which was adopted from the <a href="https://nerfies.github.io" target="_blank">Nerfies</a> project page.
You are free to borrow the of this website, we just ask that you link back to this page in the footer. <br> This website is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/" target="_blank">Creative
Commons Attribution-ShareAlike 4.0 International License</a>.
</p>
</div>
</div>
</div>
</div>
</footer>
</body>
</html>