Document the difference between tracing time and execution time

[tengyifei]

📚 Documentation

If we write a loop like

start = time.time()
for step in range(num_steps):
  run_model()
  xm.mark_step()
end = time.time()

Then end - start will only measure the tracing time. We'll need to do torch_xla.sync(wait=True) to block on device execution to measure the execution time.

We should document this in some "common FAQs/sharp edges" maybe

[sdasgup3]

@tengyifei As mark_step is synonymous with torxh_xla.sync (

xla/torch_xla/core/xla_model.py

Line 1056 in 01b5408

def mark_step(wait: bool = False, reset_scope: bool = True):

), should this issue be deprecated?

[tengyifei]

@sdasgup3 no. we can replace mark_step with sync and the same ideas still persist:

start = time.time()
for step in range(num_steps):
  run_model()
  torch_xla.sync()
end = time.time()

[sdasgup3]

@tengyifei I have added a section ## Case Study: Correctly Profiling Loops with torch_xla.sync in #9133 for the same. Please take a look.

[tengyifei]

That looks great!