[Inductor] Compiling on GPU with cpp_wrapper leads to ValueError: not enough values to unpack (expected 68, got 1)

[jon-chuang]

🐛 Describe the bug

Don't have a repro for now.

But it originates in self.codegen_with_cpp_wrapper() if self.cpp_wrapper else self.codegen()

I notice the tutorial only returns single values: https://pytorch.org/tutorials/prototype/inductor_cpp_wrapper_tutorial.html

Compiling for CUDA.

Versions

main

cc: @desertfire @jgong5

cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @muchulee8 @aakhundov @ColinPeppler

[vadimkantorov]

For instance, this fails:

This example fails in bare python too... a, b = fn() ValueError: not enough values to unpack (expected 2, got 1)

It either should replace b by *b or (1, 2,), by (1, 2,),None

[jon-chuang]

EDIT: this is actually due to mistakenly using example_inputs as the inputs to the graph. Somehow this does not correspond the the actual inputs to the graphmodule forward...

This was introduced in #114067 @desertfire

[jon-chuang]

Repro

from torch import nn
from torch._inductor import config

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(10, 10, device='cuda')
    
    def forward(self, x):
        return self.linear(x)
    

with torch.no_grad(), config.patch({"cpp_wrapper": True}):
    model = Model()
    model_opt = torch.compile(model)
    model_opt(torch.zeros(10, device="cuda"))

[desertfire]

EDIT: this is actually due to mistakenly using example_inputs as the inputs to the graph. Somehow this does not correspond the the actual inputs to the graphmodule forward...

This was introduced in #114067 @desertfire

You are right. Using real_inputs works for the AOTInductor mode, but not JIT with cpp_wrapper, and our current test does not catch that. There is something additional needs to handled in your #115053, but let me comment there.