RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

[Albert-learner]

Hello, I'm interested in your model. So I'm trying to rendering image by pretrained model.
I'm trying to render image and video at FFHQ512 dataset.
According to docs/TRAIN_EVAL.md, I understand that if I make ffhq512x512.zip by styleganv2-ada-pytorch and ffh1512_deep3dface_coeffs, then
I can render image, video and extrach mesh.

But when I apply render a image to get mpi results, this error came out.

RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

I locate pretrained model files at ml-gmpi/ckpts/gmpi_pretrained/FFHQ512.
Could you tell me why this problem happen.
Thanks for your attention.

[Xiaoming-Zhao]

Thanks a lot for your interest in our work.

Could you please provide more details about your use case, e.g., the command, GPU memory, CUDA version, and cuDNN version?

I just searched around with your error message, it may come from not-enough GPU memory, see this link. If this is the case, you could try to use small chunk_n_planes as mentioned in the README and #14

Hope this helps.

[Albert-learner]

I think my problem is not compatible with cuda and cudnn version in my device. So I will try to change those version compatible with my device and ask again for more details. Please don't close this issue, and thank you for your quick answer.

[Albert-learner]

I solve this problem by changing cudatoolkit=10.2 to cudatoolkit=11.1. Thank you for your help.

[Albert-learner]

I'm sorry, but when I apply rendering a video according to your code

Render a Video
The following command renders a video in ${OUTPUT_DIR}, along with:

video_rgb.mp4: video for the RGB rendering,
video_depth.mp4: video for the depth rendering.
conda activate gmpi && \
export PYTHONPATH=${GMPI_ROOT}:${GMPI_ROOT}/gmpi/models:$PYTHONPATH && \
python ${GMPI_ROOT}/gmpi/eval/vis/render_video.py \
--ckpt_path ${OUTPUT_DIR}/generator.pth \
--output_dir ${OUTPUT_DIR} \
--seeds ${SEED} \
--nplanes 96 \
--truncation_psi ${TRUNCATION_PSI} \
--exp_config ${OUTPUT_DIR}/config.pth \
--render_single_image 0 \
--horizontal_cam_move 1

I got this error,
TypeError: write_frames() got an unexpected keyword argument 'audio_path'
Could you tell me what is the problem??
I make rendering a video code to bash file, like this:

export GMPI_ROOT=$PWD
export DATASET_NAME=FFHQ512
export OUTPUT_DIR=${GMPI_ROOT}/ckpts/gmpi_pretrained/${DATASET_NAME}
export SEED=11
export TRUNCATION_PSI=1.0
export PYTHONPATH=${GMPI_ROOT}:${GMPI_ROOT}/gmpi/models:$PYTHONPATH && \
python ${GMPI_ROOT}/gmpi/eval/vis/render_video.py \
--ckpt_path ${OUTPUT_DIR}/generator.pth \
--output_dir ${OUTPUT_DIR} \
--seeds ${SEED} \
--nplanes 32 \
--truncation_psi ${TRUNCATION_PSI} \
--exp_config ${OUTPUT_DIR}/config.pth \
--render_single_image 0 \
--horizontal_cam_move 1

[Xiaoming-Zhao]

Do you mind posting the full traceback if possible? That would be helpful here.

And what is the version of your package imageio and imageio-ffmpeg? My guts say that this is related to the package versions as

• For imageio-ffmpeg: 0.4.3 does not have the argument audio_path (this link) while the newer version has it (this link).
• For ffmpeg: 2.9.0 does not use audio_path when calling ffmpeg (this link) while the newer version uses it (this link).

So this may come from package incompatibility. If this is the case, you could consider either downgrading the imageio or upgrading imageio-ffmpeg. FYI: the combination of imageio=2.9.0 and imageio-ffmpeg=0.4.3 is the one that I find workable. But there are definitely many other combinations that could also work.

[Albert-learner]

Thanks for your help. The problem is the version of imageio. When I see environment.yml for setting environment, it install the newest version of imageio. But when I change imageio newest version to 2.9.0, it is solved.

[Xiaoming-Zhao]

Thanks a lot for confirming the issue. Glad that it works now.

[Albert-learner]

Hello, thanks for your kind reply. I have other questions at Generative MultiPlane Image models.
In here, I'd like to see the parts of "Alpha Branch" in your paper. But when I see the code, all StyleGANv2 models came from pretrained model and the model was trained at face datasets, like FFHQ, MetFaces.

But I'd like to train my custom dataset(RealEstate10K frame images), and I guess it will not work because data domain is different.

My First Question is that I'd like to use your model at different domain datasets. In that case, I think I have to re-train StyleGANv2 to my custom datasets. Is it right??
And last Question is where is the Alpha Branch Part in your code when I evaluate FFHQ 512 dataset??

[Albert-learner]

Hello, this is Albert. And I have other questions about your model. Could you please check closed issues at your Git Repository and reply to those?? Thank you for your kind reply. Sincerely Albert. 2023년 2월 19일 (일) 오후 2:41, Xiaoming Zhao ***@***.***>님이 작성:

…

Thanks a lot for confirming the issue. Glad that it works now. — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJPACNDQOMGC4MUXXANJYLTWYGXB3ANCNFSM6AAAAAAVAFPPNA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

--

[Xiaoming-Zhao]

Hi Albert, to answer your questions:

I'd like to use your model for different domain datasets. In that case, I think I have to re-train StyleGANv2 to my custom datasets. Is it right

Not necessary. You can directly train GMPI from the scratch. At least for FFHQ, we tried to train from the scratch before. The results look good. It will just take quite a long time as there is no prior from the pre-trained checkpoints anymore.

Another caveat I want to mention: GMPI needs a camera pose for the discriminator to be conditioned on. Essentially, this means that you need to be able to somehow provide camera poses for images from RealEstate10K. Frankly speaking, how to adapt GMPI to and whether GMPI will work well on such indoor scene data without further modification/adaptation is still an open research problem. Only experiments could tell.

where is the Alpha Branch Part in your code

Assume you use the model variant that conditions on normalized depth as mentioned here. Then this line could be a good starting point for you to understand how alphas are computed, i.e., checking how cond_x is computed:

ml-gmpi/gmpi/models/networks/networks_cond_on_pos_enc.py

Line 880 in 672294b

cond_x = cond_x.unsqueeze(1).expand(-1, self.n_planes, -1, -1, -1).reshape((bs * self.n_planes, -1, self.resolution, self.resolution))

As the variable cond_x will later be used to generate the alpha maps:

ml-gmpi/gmpi/models/networks/networks_cond_on_pos_enc.py

Line 969 in 672294b

cur_alpha = self.toalpha(cond_x, w_alpha, fused_modconv=fused_modconv, splitted=True, n_planes=self.n_planes)

Hope these may help.