Dynamo inaccurate line number in backtrace

[ezyang]

Steps to reproduce:

1. Build 86c182c6cd3a39e02a3f3575510de5b3d2c98acd of pytorch/pytorch (current tip of Symbolic shapes pytorch#84246)
2. Normal torchdynamo OK (tested 2e6737a)
3. Run TORCH_SHOW_CPP_STACKTRACES=1 TORCHDYNAMO_DYNAMIC_SHAPES=1 AOT_DYNAMIC_SHAPES=1 TORCH_MONKEYPATCH_GETITEM=1 python benchmarks/torchbench.py --only hf_T5 --accuracy --backend aot_eager --training

This errors with:

  File "/home/ezyang/local/pytorch-tmp-env/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py", line 892, in forward  
    def forward(                                                  
  File "/home/ezyang/local/pytorch-tmp-env/lib/python3.9/site-packages/transformers/modeling_utils.py", line 824, in get_extended_attention_mask                                                       
    extended_attention_mask = ModuleUtilsMixin.create_extended_attention_mask_for_decoder(                                           
  File "/home/ezyang/local/pytorch-tmp-env/lib/python3.9/site-packages/transformers/modeling_utils.py", line 774, in create_extended_attention_mask_for_decoder                                        
    batch_size, seq_length = input_shape                                                                                             
TypeError: cannot unpack non-iterable int object   

Notice that the first line refers to 892, aka the function definition

      def forward(
          self,
          input_ids=None,
          attention_mask=None,
          encoder_hidden_states=None,
          encoder_attention_mask=None,
          inputs_embeds=None,
          head_mask=None,
          cross_attn_head_mask=None,
          past_key_values=None,
          use_cache=None,
          output_attentions=None,
          output_hidden_states=None,
          return_dict=None,
      ):  

This is not expected. I expect the reported line number to be 958, aka where get_extended_attention_mask is called:

>>        # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
>>        # ourselves in which case we just need to make it broadcastable to all heads.
>>        extended_attention_mask = self.get_extended_attention_mask(attention_mask, input_shape)

(the dynamo error itself is unlikely to be a dynamo problem and is some transient problem with the symbolic shapes stack)

[anijain2305]

Cc @williamwen42

[jansel]

Does turning off torchdynamo.config.dead_code_elimination fix this issue?

torchdynamo/torchdynamo/convert_frame.py

Line 374 in 7350f88

if config.dead_code_elimination:

If so we may just need to update remove_pointless_jumps and remove_dead_code to preserve line numbers.

Each bytecode instruction has a inst.starts_line property that is an integer for the first instruction of each line, and None otherwise. If we delete the first instruction then we will clobber the line numbers for that one line.

I'd suggest adding a pass that fills in inst.starts_line for every instruction at the start. Then strips away the redundant ones right before we assemble output byteocde.

We have some similar stuff here:

torchdynamo/torchdynamo/bytecode_transformation.py

Line 345 in 7350f88

devirtualize_jumps(instructions)

[ezyang]

Disabling DCE did not fix the issue

  File "/home/ezyang/local/pytorch-tmp-env/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py", line 892, in forward
    def forward(
  File "/home/ezyang/local/pytorch-tmp-env/lib/python3.9/site-packages/transformers/modeling_utils.py", line 824, in get_extended_attention_mask
    extended_attention_mask = ModuleUtilsMixin.create_extended_attention_mask_for_decoder(
  File "/home/ezyang/local/pytorch-tmp-env/lib/python3.9/site-packages/transformers/modeling_utils.py", line 774, in create_extended_attention_mask_for_decoder
    batch_size, seq_length = input_shape