Skip to content
/ evan Public

🔍 python package to evaluate GANs for video generation

Notifications You must be signed in to change notification settings

raahii/evan

Repository files navigation

Library and CLI tool for evaluating video GANs

build

"Evan" automates GAN evaluation for video generation. The library gives two advantages for you.

  1. You don't need to prepare the Inception Model and process your samples for evaluation.
  2. You don't need to find source codes or write algorithm of each GAN metric.

Now, evan supports following metrics.

  • Inception Score [1]
  • Frechet Inception Distace [2]
  • Precision and Recall for Distributions [3]

Installation

Required

$ pip install evan

Example

See examples directory or docs for details.

import torch
from pathlib import Path
import skvideo.io
import evan

gen = YourGenerater()
N = 5000
batchsize = 100
temp = tempfile.TemporaryDirectory()
temp_dir = Path(temp.name)
fps = 30

# generate video samples by your GAN and
# save them into a directory in .mp4
for batch in range(0, N, batchsize):
	xs = gen.generate(batchsize) # (B, T, H, W, C)
	for i, x in enumerate(xs):
		path = str(temp_dir / f"{i}.mp4")
		writer = skvideo.io.FFmpegWriter(path, inputdict={"-r": str(fps)})
		for frame in video:
        writer.writeFrame(frame)

# compute Inception Score by using evan
score = evan.compute_inception_score(
	temp_dir, batchsize=batchsize, verbose=True
)
# >> found 10000 samples.
# >> converting videos into conv features using inception model (on gpu)...
#     100%|█████████████████████████████████| 10000/10000 [XX:XX<XX:XX, XX.XX s/it]
# >> computing IS...
#     generated samples: '/var/folders/k4/xxx/probs.npy'

print(score)
temp.cleanup()

CLI

❯ evan -h
usage: evan [-h] {compute,plot} ...

a tool for evaluation video GANs.

optional arguments:
  -h, --help      show this help message and exit

subcommands:
  command names.

  {compute,plot}
    compute       compute evaluation score.
    plot          visualize evaluation result.

Details

Evaluation flow

WIP

Inception Model

WIP

Reference