Cannot update ipadapter weight after compiling the graph in ComfyUI

[Gutianpei]

Describe the bug

The workflow contains animatediff node and ipadapter node. After compiling the graph, changing the ipadapter weight will not work (the video is still generated with the weight when the graph first get compiled)

Your environment

Ubuntu 20.04

OneDiff git commit id

2249010

OneFlow version info

libibverbs not available, ibv_fork_init skipped
path: ['/home/ubuntu/miniconda3/envs/comfy/lib/python3.10/site-packages/oneflow']
version: 0.9.1.dev20240420+cu121
git_commit: 665bcf8
cmake_build_type: Release
rdma: True
mlir: True
enterprise: False

How To Reproduce

weight_ipadapter.json
After running the workflow, changing the weight in the node "Batch Value Schedule". It's expected to generate video according to the new weights but instead the video is never updated. One extreme case is set a weight to 100 after compiling, we should observe very unreasonable result.

The complete error message

No error message noted.

Additional context

Dependency version is followed this pr: #879
ComfyUI-AnimateDiff-Evolved commit id: f9e0343f4c4606ee6365a9af4a7e16118f1c45e1

[ccssu]

Debug found that the inability to update is due to an issue with the torch workflow：

The workflow weight did not change to a list, but rather to a float, as follows：
https://github.com/cubiq/ComfyUI_IPAdapter_plus/blob/d33265adf72974213fd4ea9bdbba1d0f176e7658/CrossAttentionPatch.py#L148-L151

# ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/CrossAttentionPatch.py:161
    else:
        out_ip = optimized_attention(q, ip_k, ip_v, extra_options["n_heads"])
        import pdb; pdb.set_trace()
        out_ip = out_ip * weight # I'm doing this to get the same results as before

  /data/home/fengwen/quant/ComfyUI/comfy/ldm/modules/attention.py(456)forward()
-> return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
  /data/home/fengwen/quant/ComfyUI/comfy/ldm/modules/diffusionmodules/util.py(191)checkpoint()
-> return func(*inputs)
  /data/home/fengwen/quant/ComfyUI/comfy/ldm/modules/attention.py(554)_forward()
-> n = attn2_replace_patch[block_attn2](n, context_attn2, value_attn2, extra_options)
  /data/home/fengwen/quant/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/CrossAttentionPatch.py(38)__call__()
-> out = out + callback(out, q, k, v, extra_options, **self.kwargs[i])
> /data/home/fengwen/quant/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/CrossAttentionPatch.py(164)ipadapter_attention()
-> out_ip = out_ip * weight # I'm doing this to get the same results as before
(Pdb) p 
*** SyntaxError: invalid syntax
(Pdb) p weight
1.0

[ccssu]

please try it test case @Gutianpei https://github.com/siliconflow/onediff/tree/3aad30db09d13a22c75fee1f11230fcb8562de6a/onediff_comfy_nodes/modules/oneflow/hijack_ipadapter_plus#ipadapter_weights

[Gutianpei]

please try it test case @Gutianpei https://github.com/siliconflow/onediff/tree/3aad30db09d13a22c75fee1f11230fcb8562de6a/onediff_comfy_nodes/modules/oneflow/hijack_ipadapter_plus#ipadapter_weights

The link for the workflow is in the private repo, would you mind to share a public link? Thanks!

[Gutianpei]

I've tested that changing weights is working, thanks for the help! closing this issue