Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Colab compatibility improvements #87

Merged
merged 3 commits into from
Jun 19, 2020
Merged

Colab compatibility improvements #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d7c0520
Ensure Google Colab GPU Compatibility
AlphaGit Jun 7, 2020
f9073d9
Supressing compiler warnings
AlphaGit Jun 7, 2020
e70d6d5
Updated documentation
AlphaGit Jun 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
214 changes: 133 additions & 81 deletions Colab_DAIN.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,16 @@
"colab_type": "text"
},
"source": [
"Credits to original Colab file: \n",
"https://github.com/baowenbo/DAIN/issues/44\n",
"*DAIN Colab, v1.4.0*\n",
"\n",
"My fork:\n",
"https://github.com/styler00dollar/DAIN\n",
"Based on the [original Colab file](https://github.com/baowenbo/DAIN/issues/44) by btahir. \n",
"\n",
"Enhancement by [Styler00Dollar](https://github.com/styler00dollar) aka \"sudo rm -rf / --no-preserve-root#8353\" on discord and [Alpha](https://github.com/AlphaGit). Please do not run this command in your linux terminal. It's rather meant as a joke.\n",
"Enhancements by [Styler00Dollar](https://github.com/styler00dollar) aka \"sudo rm -rf / --no-preserve-root#8353\" on discord and [Alpha](https://github.com/AlphaGit), (Alpha#6137 on Discord). Please do not run this command in your linux terminal. It's rather meant as a joke.\n",
"\n",
"[Styler00Dollar's fork](https://github.com/styler00dollar/DAIN) / [Alpha's fork](https://github.com/AlphaGit/DAIN)\n",
"\n",
"A simple guide:\n",
"- Upload the ` .ipynb` file to your Google Colab.\n",
"- Upload this ` .ipynb` file to your Google Colab.\n",
"- Create a folder inside of Google Drive named \"DAIN\"\n",
"- Change the configurations in the next cell\n",
"- Run cells one by one\n",
Expand All @@ -52,7 +52,7 @@
"- Adding configuration to select speed\n",
"- Detect scenes to avoid interpolating scene-changes\n",
"- Auto-resume\n",
"- Copy `start_frame` - `end_frame` audio from original input to final output"
"- Copy `start_frame` - `end_frame` audio from original input to final output\n"
]
},
{
Expand All @@ -63,51 +63,49 @@
"colab": {}
},
"source": [
"################# Configurations ############################\n",
"################# Required Configurations ############################\n",
"\n",
"# Use the values in here to configure what you'd like DAIN to do.\n",
"#@markdown # Required Configuration\n",
"#@markdown Use the values in here to configure what you'd like DAIN to do.\n",
"\n",
"# Input file: Path (relative to the root of your Google Drive) to the input file.\n",
"# For instance, if you save your \"example.mkv\" file in your Google Drive, inside a \"videos\" folder, the path would be:\n",
"# videos/example.mkv. Currenly videos and gifs are supported.\n",
"INPUT_FILEPATH = \"DAIN/input.mp4\"\n",
"#@markdown ## Input file\n",
"#@markdown Path (relative to the root of your Google Drive) to the input file. For instance, if you save your `example.mkv` file in your Google Drive, inside a `videos` folder, the path would be: `videos/example.mkv`. Currenly videos and gifs are supported.\n",
"INPUT_FILEPATH = \"DAIN/input.mp4\" #@param{type:\"string\"}\n",
"\n",
"# Output file path: path (relative to the root of your Google Drive) for the output file. It will also determine the\n",
"# filetype in the destination. MP4 is recommended.\n",
"OUTPUT_FILE_PATH = \"DAIN/output.mp4\"\n",
"#@markdown ## Output file\n",
"#@markdown Output file path: path (relative to the root of your Google Drive) for the output file. It will also determine the filetype in the destination. `.mp4` is recommended for video input, `.gif` for gif inputs.\n",
"OUTPUT_FILE_PATH = \"DAIN/output.mp4\" #@param{type:\"string\"}\n",
"\n",
"################# Optional configurations ############################\n",
"\n",
"# Target FPS = how many frames per second should the result have. This will determine how many intermediate images are\n",
"# interpolated.\n",
"TARGET_FPS = 60\n",
"\n",
"# Frame input directly\n",
"# Use a path that is in your GDrive if you already have the list of frames in the format 00001.png, 00002.png, etc.\n",
"# Your GDrive is located at `/content/gdrive/My Drive/`\n",
"FRAME_INPUT_DIR = '/content/DAIN/input_frames'\n",
"\n",
"# Frame output directory\n",
"# Use a location in your GDrive if you want the generated frames stored to your Google Drive.\n",
"# Your GDrive is located at `/content/gdrive/My Drive/DAIN/tmp`\n",
"FRAME_OUTPUT_DIR = '/content/DAIN/output_frames'\n",
"\n",
"# Seamless playback\n",
"# Creates a seamless loop by using the first frame as last one as well. Set this to True this if loop is intended.\n",
"SEAMLESS = False\n",
"\n",
"# Resize hotfix\n",
"# DAIN frames are a bit \"shifted / smaller\" compared to original input frames. This can partly be mitigated with resizing\n",
"# DAIN frames to the resolution +2px and cropping the result to the original resoultion with the starting point (1,1).\n",
"# Without this fix, DAIN tends to make \"vibrating\" output and it is pretty noticible with static elements like text.\n",
"# This hotfix tries to make such effects less visible for a smoother video playback. I do not know what DAINAPP\n",
"# uses as a fix for this problem, but the original does show such behaviour with the default test images. More advanced\n",
"# users can change the interpolation method. The methods cv2.INTER_CUBIC and cv2.INTER_LANCZOS4 are recommended.\n",
"# The current default value is cv2.INTER_LANCZOS4.\n",
"RESIZE_HOTFIX = True\n",
"\n",
"# Auto-delete output PNG dir after ffmpeg video creation.\n",
"AUTO_REMOVE = True"
"#@markdown # Optional Configuration\n",
"#@markdown Parameters below can be left with their defaults, but feel free to adapt them to your needs.\n",
"\n",
"#@markdown ## Target FPS\n",
"#@markdown how many frames per second should the result have. This will determine how many intermediate images are interpolated.\n",
"TARGET_FPS = 60 #@param{type:\"number\"}\n",
"\n",
"#@markdown ## Frame input directory\n",
"#@markdown A path, relative to your GDrive root, where you already have the list of frames in the format 00001.png, 00002.png, etc.\n",
"FRAME_INPUT_DIR = '/content/DAIN/input_frames' #@param{type:\"string\"}\n",
"\n",
"#@markdown ## Frame output directory\n",
"#@markdown A path, relative to your GDrive root, where you want the generated frame.\n",
"FRAME_OUTPUT_DIR = '/content/DAIN/output_frames' #@param{type:\"string\"}\n",
"\n",
"#@markdown ## Seamless playback\n",
"#@markdown Creates a seamless loop by using the first frame as last one as well. Set this to True this if loop is intended.\n",
"SEAMLESS = False #@param{type:\"boolean\"}\n",
"\n",
"#@markdown ## Resize hotfix\n",
"#@markdown DAIN frames are a bit \"shifted / smaller\" compared to original input frames. This can partly be mitigated with resizing DAIN frames to the resolution +2px and cropping the result to the original resoultion with the starting point (1,1). Without this fix, DAIN tends to make \"vibrating\" output and it is pretty noticible with static elements like text.\n",
"#@markdown\n",
"#@markdown This hotfix tries to make such effects less visible for a smoother video playback. I do not know what DAINAPP uses as a fix for this problem, but the original does show such behaviour with the default test images. More advanced users can change the interpolation method. The methods cv2.INTER_CUBIC and cv2.INTER_LANCZOS4 are recommended. The current default value is cv2.INTER_LANCZOS4.\n",
"RESIZE_HOTFIX = True #@param{type:\"boolean\"}\n",
"\n",
"#@markdown ## Auto-remove PNG directory\n",
"#@markdown Auto-delete output PNG dir after ffmpeg video creation. Set this to `False` if you want to keep the PNG files.\n",
"AUTO_REMOVE = True #@param{type:\"boolean\"}"
],
"execution_count": 0,
"outputs": []
Expand Down Expand Up @@ -142,11 +140,6 @@
"# 16GB: Can handle 720p. 1080p will procude an out-of-memory error. \n",
"# 8GB: Can handle 480p. 720p will produce an out-of-memory error.\n",
"\n",
"# P100: 16GB (Works)\n",
"# T4: 16GB [RuntimeError: CUDA call failed]\n",
"# P4: 8GB (Works)\n",
"# K80: 8GB (Not tested)\n",
"\n",
"!nvidia-smi --query-gpu=gpu_name,driver_version,memory.total --format=csv"
],
"execution_count": 0,
Expand All @@ -169,38 +162,98 @@
{
"cell_type": "code",
"metadata": {
"id": "UeaU8um5-2NS",
"id": "XunYRBtyPZf_",
"colab_type": "code",
"colab": {}
},
"source": [
"# Install known used versions of PyTorch and SciPy\n",
"!pip install torch==1.4.0+cu100 torchvision==0.5.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html\n",
"!pip install scipy==1.1.0\n",
"\n",
"from IPython.display import clear_output\n",
"!git clone https://github.com/baowenbo/DAIN /content/DAIN\n",
"\n",
"!pip install scipy==1.1.0"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "-mRVRXIGPk9I",
"colab_type": "code",
"colab": {}
},
"source": [
"# Clone DAIN sources\n",
"# TODO Bring back original repo\n",
"#!git clone https://github.com/baowenbo/DAIN /content/DAIN\n",
"%cd /content\n",
"!rm -rf DAIN\n",
"!git clone -b colab-compatibility --depth 1 https://github.com/AlphaGit/DAIN /content/DAIN\n",
"%cd /content/DAIN\n",
"!git log -1"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "e5AHGetTRacZ",
"colab_type": "code",
"colab": {}
},
"source": [
"# This takes a while. Just wait. ~15 minutes.\n",
"# Building DAIN.\n",
"%cd /content/DAIN/my_package/\n",
"!./build.sh\n",
"print(\"Building #1 done.\")\n",
"\n",
"print(\"Building #1 done.\")"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "UeaU8um5-2NS",
"colab_type": "code",
"colab": {}
},
"source": [
"# Wait again. ~5 minutes.\n",
"# Building DAIN PyTorch correlation package.\n",
"%cd /content/DAIN/PWCNet/correlation_package_pytorch1_0\n",
"!./build.sh\n",
"print(\"Building #2 done.\")\n",
"\n",
"print(\"Building #2 done.\")"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "InjqMYIyXCZs",
"colab_type": "code",
"colab": {}
},
"source": [
"# Downloading pre-trained model\n",
"%cd /content/DAIN\n",
"!mkdir model_weights\n",
"!wget -O model_weights/best.pth http://vllab1.ucmerced.edu/~wenbobao/DAIN/best.pth\n",
"\n",
"!wget -O model_weights/best.pth http://vllab1.ucmerced.edu/~wenbobao/DAIN/best.pth"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "FrdWsqbvXEid",
"colab_type": "code",
"colab": {}
},
"source": [
"!CUDA_VISIBLE_DEVICES=0\n",
"\n",
"!sudo apt-get install imagemagick imagemagick-doc\n",
"\n",
"print(\"Finished installing dependencies.\")"
],
"execution_count": 0,
Expand Down Expand Up @@ -246,6 +299,7 @@
"%shell ffmpeg -i '/content/DAIN/{filename}' '{FRAME_INPUT_DIR}/%05d.png'\n",
"\n",
"png_generated_count_command_result = %shell ls '{FRAME_INPUT_DIR}' | wc -l\n",
"from IPython.display import clear_output\n",
"clear_output()\n",
"\n",
"pngs_generated_count = int(png_generated_count_command_result.output.strip())\n",
Expand Down Expand Up @@ -328,44 +382,42 @@
"source": [
"# Create video\n",
"%cd {FRAME_OUTPUT_DIR}\n",
"%shell ffmpeg -y -r {TARGET_FPS} -f image2 -pattern_type glob -i '*.png' '/content/gdrive/My Drive/{OUTPUT_FILE_PATH}'\n",
"if(AUTO_REMOVE==True):\n",
" !rm -rf {FRAME_OUTPUT_DIR}/*"
"%shell ffmpeg -y -r {TARGET_FPS} -f image2 -pattern_type glob -i '*.png' '/content/gdrive/My Drive/{OUTPUT_FILE_PATH}'"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "uCBHIXNN-JYu",
"id": "UF5TEo5N374o",
"colab_type": "code",
"colab": {}
},
"source": [
"# [Experimental] Create video with sound\n",
"# Only run this, if the original had sound.\n",
"%cd {FRAME_OUTPUT_DIR}\n",
"%shell ffmpeg -i '/content/DAIN/{filename}' -acodec copy output-audio.aac\n",
"%shell ffmpeg -y -r {TARGET_FPS} -f image2 -pattern_type glob -i '*.png' -i output-audio.aac -shortest '/content/gdrive/My Drive/{OUTPUT_FILE_PATH}'\n",
"\n",
"if (AUTO_REMOVE):\n",
" !rm -rf {FRAME_OUTPUT_DIR}/*\n",
" !rm -rf output-audio.aac"
"if(AUTO_REMOVE):\n",
" !rm -rf {FRAME_OUTPUT_DIR}/*"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "8tTXpWkRBm8T",
"id": "uCBHIXNN-JYu",
"colab_type": "code",
"colab": {}
},
"source": [
"# Manually removing output PNG dir (before working with another file)\n",
"!rm -rf {FRAME_OUTPUT_DIR}/*"
"# [Experimental] Create video with sound\n",
"# Only run this, if the original had sound.\n",
"%cd {FRAME_OUTPUT_DIR}\n",
"%shell ffmpeg -i '/content/DAIN/{filename}' -acodec copy output-audio.aac\n",
"%shell ffmpeg -y -r {TARGET_FPS} -f image2 -pattern_type glob -i '*.png' -i output-audio.aac -shortest '/content/gdrive/My Drive/{OUTPUT_FILE_PATH}'\n",
"\n",
"if (AUTO_REMOVE):\n",
" !rm -rf {FRAME_OUTPUT_DIR}/*\n",
" !rm -rf output-audio.aac"
],
"execution_count": 0,
"outputs": []
Expand Down
1 change: 1 addition & 0 deletions PWCNet/correlation_package_pytorch1_0/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
echo "Need pytorch>=1.0.0"
source activate pytorch1.0.0

export PYTHONPATH=$PYTHONPATH:$(pwd)/../../my_package

rm -rf build *.egg-info dist
python setup.py install
11 changes: 1 addition & 10 deletions PWCNet/correlation_package_pytorch1_0/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,7 @@
from setuptools import setup, find_packages
from torch.utils.cpp_extension import BuildExtension, CUDAExtension

cxx_args = ['-std=c++11']

nvcc_args = [
'-gencode', 'arch=compute_50,code=sm_50',
'-gencode', 'arch=compute_52,code=sm_52',
'-gencode', 'arch=compute_60,code=sm_60',
'-gencode', 'arch=compute_61,code=sm_61'
# '-gencode', 'arch=compute_70,code=sm_70',
# '-gencode', 'arch=compute_70,code=compute_70'
]
from compiler_args import nvcc_args, cxx_args

setup(
name='correlation_cuda',
Expand Down
10 changes: 8 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,9 +229,15 @@ Then test the new model by executing:
This is a modification of DAIN that allows the usage of Google Colab and is able to do a full demo interpolation from a source video to a target video.

Original Notebook File by btahir can be found [here](https://github.com/baowenbo/DAIN/issues/44).
This is a modification by [Styler00Dollar](https://github.com/styler00dollar) and [Alpha](https://github.com/AlphaGit).

Simply upload the `Colab_DAIN.ipynb` file to your Google Drive or use this [link](https://colab.research.google.com/drive/1gzsfDV_MIdehr7Y8ZzWjTuW-mMZRP4Vy).
To use the Colab, follow these steps:

- Download the `Colab_DAIN.ipynb` file ([link](https://raw.githubusercontent.com/baowenbo/DAIN/master/Colab_DAIN.ipynb)).
- Visit Google Colaboratory ([link](https://colab.research.google.com/))
- Select the "Upload" option, and upload the `.ipynb` file
- Start running the cells one by one, following the instructions.

Colab file authors: [Styler00Dollar](https://github.com/styler00dollar) and [Alpha](https://github.com/AlphaGit).

### Contact
[Wenbo Bao](mailto:bwb0813@gmail.com); [Wei-Sheng (Jason) Lai](mailto:phoenix104104@gmail.com)
Expand Down
11 changes: 1 addition & 10 deletions my_package/DepthFlowProjection/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,7 @@
from setuptools import setup, find_packages
from torch.utils.cpp_extension import BuildExtension, CUDAExtension

cxx_args = ['-std=c++11']

nvcc_args = [
'-gencode', 'arch=compute_50,code=sm_50',
'-gencode', 'arch=compute_52,code=sm_52',
'-gencode', 'arch=compute_60,code=sm_60',
'-gencode', 'arch=compute_61,code=sm_61'
# '-gencode', 'arch=compute_70,code=sm_70',
# '-gencode', 'arch=compute_70,code=compute_70'
]
from compiler_args import nvcc_args, cxx_args

setup(
name='depthflowprojection_cuda',
Expand Down
11 changes: 1 addition & 10 deletions my_package/FilterInterpolation/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,7 @@
from setuptools import setup, find_packages
from torch.utils.cpp_extension import BuildExtension, CUDAExtension

cxx_args = ['-std=c++11']

nvcc_args = [
'-gencode', 'arch=compute_50,code=sm_50',
'-gencode', 'arch=compute_52,code=sm_52',
'-gencode', 'arch=compute_60,code=sm_60',
'-gencode', 'arch=compute_61,code=sm_61'
# '-gencode', 'arch=compute_70,code=sm_70',
# '-gencode', 'arch=compute_70,code=compute_70'
]
from compiler_args import nvcc_args, cxx_args

setup(
name='filterinterpolation_cuda',
Expand Down