[Dynamic Shape] Dynamic size can not be printed

[JackCaoG]

🚀 Feature

Support printing a dynamic size.

Motivation

This is a common user behavior, we need to provide some kind of output.

Pitch

I think we agreed that it should print the upper_bound but correct me if I am wrong.

Repo

PyTorch and PyTorch/XLA both at nightly.

import torch
import torch_xla.core.xla_model as xm

import os
os.environ["XLA_EXPERIMENTAL"] = "nonzero:masked_select:masked_scatter"

dev = xm.xla_device()
num_element = 10
t1 = torch.zeros([num_element], device=dev)
t1[0] = 1
t2 = torch.nonzero(t1)
print(t2.size())

error

(Pdb) print(t2.size())
*** RuntimeError: AttributeError: 'torch._C._SymNode' object has no attribute 'str'

At:
  /pytorch/torch/__init__.py(238): __repr__
  <stdin>(1): <module>
  /root/anaconda3/envs/pytorch/lib/python3.7/pdb.py(374): default
  /root/anaconda3/envs/pytorch/lib/python3.7/cmd.py(216): onecmd
  /root/anaconda3/envs/pytorch/lib/python3.7/pdb.py(416): onecmd
  /root/anaconda3/envs/pytorch/lib/python3.7/cmd.py(138): cmdloop
  /root/anaconda3/envs/pytorch/lib/python3.7/pdb.py(319): _cmdloop
  /root/anaconda3/envs/pytorch/lib/python3.7/pdb.py(350): interaction
  /root/anaconda3/envs/pytorch/lib/python3.7/pdb.py(259): user_line
  /root/anaconda3/envs/pytorch/lib/python3.7/bdb.py(112): dispatch_line
  /root/anaconda3/envs/pytorch/lib/python3.7/bdb.py(88): trace_dispatch
  test/test_dynamic_fail.py(17): <module>

@miladm @vanbasten23 @Krovatkin @wconstab

[JackCaoG]

both IR and HLO seems fine, so I guess we need to plum some python API

(Pdb) print(torch_xla._XLAC._get_xla_tensors_text([t2]))
IR {
  %0 = s64[<=10,1]{1,0} xla::device_data(), location=@No Python Frames:-1, device=CPU:0, ROOT=0
}

(Pdb) print(torch_xla._XLAC._get_xla_tensors_hlo([t2]))
HloModule IrToHlo.3, entry_computation_layout={(s64[<=10,1]{1,0})->(s64[<=10,1]{1,0})}

ENTRY %IrToHlo.3 (p0.1: s64[<=10,1]) -> (s64[<=10,1]) {
  %p0.1 = s64[<=10,1]{1,0} parameter(0), metadata={op_type="xla__device_data" op_name="xla__device_data" source_file="@No Python Frames" source_line=-1}
  ROOT %tuple.2 = (s64[<=10,1]{1,0}) tuple(s64[<=10,1]{1,0} %p0.1)
}

[vanbasten23]

I wonder if we need to add the test in this issue to the code base. If so, should we add it to pytorch or pytorch/xla?

[JackCaoG]

we should add a test to https://github.com/pytorch/xla/blob/master/test/test_dynamic_shapes.py