36 Dynamo test are failing with "None missing".

[zou3519]

Repro

PYTORCH_TEST_WITH_DYNAMO=1 pytest test_scatter_gather_ops.py -v -k test_scatter__reductions_cpu_complex64

You will need to remove the skip or expectedFailure before running the repro command.
This may be just removing a line in
dynamo_test_failures.py

Failing tests

Here's a comprehensive list of tests that fail (as of this issue) with the above message:

Click me

``` test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter__reductions_cpu_complex64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter__reductions_cpu_float16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter__reductions_cpu_float32 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_bfloat16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_complex128 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_complex64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_float16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_float32 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_float64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_int16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_int32 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_int64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_int8 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_mean_cpu_uint8 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_bfloat16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_complex128 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_complex64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_float16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_float32 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_float64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_int16 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_int32 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_int64 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_int8 test_scatter_gather_ops.py::TestScatterGatherCPU.test_scatter_reduce_sum_cpu_uint8 nn/test_init.py::TestNNInit.test_constant nn/test_init.py::TestNNInit.test_dirac_only_works_on_3_4_5d_inputs nn/test_init.py::TestNNInit.test_eye nn/test_init.py::TestNNInit.test_eye_only_works_on_2d_inputs nn/test_init.py::TestNNInit.test_kaiming_normal_errors_on_inputs_smaller_than_2d nn/test_init.py::TestNNInit.test_kaiming_uniform_errors_on_inputs_smaller_than_2d nn/test_init.py::TestNNInit.test_ones_and_zeros nn/test_init.py::TestNNInit.test_sparse_only_works_on_2d_inputs nn/test_init.py::TestNNInit.test_xavier_normal_errors_on_inputs_smaller_than_2d nn/test_init.py::TestNNInit.test_xavier_uniform nn/test_init.py::TestNNInit.test_xavier_uniform_errors_on_inputs_smaller_than_2d ```

cc @ezyang @msaroufim @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng

[laithsakka]

Looking at this, it looks like someone has looked at this before
I see a comment describing the issue
1e89a56

   # Really annoying intersection of specialization and RandomValueSource
    # If we get a RandomValueSource with a single element tensor, we should return a ConstantVariable like other
    # unspects... but if we do, we break the bytecode assumptions and guards will not work as we will be referring
    # to a name from a source that is not there. If we call .item() and take the wrapped_value out, where we do
    # wrapped_value = wrapped_value.item() where we send unspec down to wrap_fx_proxy, this test passes and then
    # some models fail on missing codegen.tx.output.random_values_var. If we let the tensor value go into wrap as
    # it is, this test fails.
    # The real solution here is to rewrite RandomValueSource and all the codegen it does from the ground up.

[laithsakka]

Ok one interesting thing the problem always happen in the second innovation of the function
if self._test_scatter_base is called only once there is no issue.

@dtypes(torch.float16, torch.float32, torch.complex64)
def test_scatter__reductions(self, device, dtype):
    for reduction in ("add", "multiply"):
        self._test_scatter_base(torch.Tensor.scatter_, device=device, dtype=dtype,
                                is_scalar=False, reduction=reduction)
        self._test_scatter_base(torch.Tensor.scatter_, device=device, dtype=dtype,
                                is_scalar=False, reduction=reduction)

[laithsakka]

I was able to simplify the repo to

     def test_rand_new(self):
        @torch.compile(backend="eager")
        def core():
            idx_size = [10]
            idx_size[random.randint(0, 1)] = random.randint(1, 9)
            t= tuple(idx_size)

            # If I remove this line  or the line after I get IndexError: list assignment index out of range
            # instread of
            src_size = [random.randint(1, 5) + s for s in idx_size]
            idx = torch.empty(t)
        core()
        core()

Note:

1. core need to be called twice error happen at second call to core!

[laithsakka]

Compiled function is not the same across iterations first time we call core() its specialized for some inputs

TRACED GRAPH
 ===== __compiled_fn_3 =====
 /data/users/lsakka/pytorch/torch/fx/_lazy_graph_module.py class GraphModule(torch.nn.Module):
    def forward(self):
        # File: /data/users/lsakka/pytorch/test/dynamo/test_functions.py:1806 in torch_dynamo_resume_in_core_at_1805, code: idx = torch.empty(t)
        idx = torch.empty((8,))
        return ()

When we call it again it tries to access some args that do not exisit!

      TRACED GRAPH
===== __compiled_fn_4 =====
/data/users/lsakka/pytorch/torch/fx/_lazy_graph_module.py class GraphModule(torch.nn.Module):
  def forward(self, L_idx_size_0_ : torch.SymInt, random_value_0 : torch.Tensor):
      l_idx_size_0_ = L_idx_size_0_
      
      # File: /data/users/lsakka/pytorch/test/dynamo/test_functions.py:1805 in <listcomp>, code: src_size = [random.randint(1, 5) + s for s in idx_size]
      add = random_value_0 + l_idx_size_0_;  random_value_0 = None
      
      # File: /data/users/lsakka/pytorch/test/dynamo/test_functions.py:1806 in torch_dynamo_resume_in_core_at_1800, code: idx = torch.empty(t)
      idx = torch.empty((l_idx_size_0_,));  l_idx_size_0_ = None
      return ()
      ```

[laithsakka]

it sounds to me that specialization for some fixed input random values makes the function prone to loosing inputs information somehow. its just a theory trying to understand it better

[zou3519]

This might be automatic dynamic shapes? The first time we run a funciton, if there are int inputs, we specialize on it. The next iterations, if we notice the int has changed, we will treat it as a SymNode (and not specialize on it). Automatic dynamic shapes is expected but the error is not

[laithsakka]

One thing i noticed if i rewrite :

            idx_size[random.randint(0, 1)] = random.randint(1, 9)

As

          randVal1 =random.randint(0, 0)
           randVal2 =random.randint(1, 8)
           # x = randVal1+randVal2
           idx_size[randVal1] = randVal2
     ```      
 
then error is gone !

[laithsakka]

This might be automatic dynamic shapes? The first time we run a funciton, if there are int inputs, we specialize on it. The next iterations, if we notice the int has changed, we will treat it as a SymNode (and not specialize on it). Automatic dynamic shapes is expected but the error is not

running it with dynamic=False remove the issue so its related to dynamism potentially

  @torch.compile(backend="eager", dynamic=False)
        def core():
            idx_size = [10]
            # randVal1 =
            # randVal2 =random.randint(1, 8)
            # # x = randVal1+randVal2
            # idx_size[randVal1] = randVal2
            idx_size[random.randint(0, 0)] = random.randint(1, 8)

            t= tuple(idx_size)

            # If I remove this line  or the line after I get IndexError: list assignment index out of range
            # instread of
            src_size = [random.randint(1, 5) + s for s in idx_size]
            idx = torch.empty(t)