Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trace page in tensorboard remains empty #805

Open
robogast opened this issue Sep 15, 2023 · 3 comments
Open

Trace page in tensorboard remains empty #805

robogast opened this issue Sep 15, 2023 · 3 comments
Labels
bug Something isn't working plugin PyTorch Profiler TensorBoard Plugin related

Comments

@robogast
Copy link

robogast commented Sep 15, 2023

The trace view is one of my most important tools for profiling pytorch programs, but for the last few days I cannot manage to get it to display anything:
image

I can see that one of the processes of tensorboard is in 'Zombie' mode:
image

Things I've tried to fix this crash:

  • Use toch_tb_profiler version 0.4.1 and 0.4.3
  • Use tensorboard version 2.11 and 2.14
  • I've checked that the plugin.py::trace_route function returns 'as expected' (or at least doesn't crash)

I'll update this issue as I investigate further

@anupambhatnagar anupambhatnagar added the plugin PyTorch Profiler TensorBoard Plugin related label Oct 5, 2023
@aaronenyeshi aaronenyeshi added the bug Something isn't working label Oct 6, 2023
@sunghyunjun
Copy link

I don't know exactly but I had same problem. And I found this issue occurs when trace.json is lager than 500MB. I manually cut last part of trace.json to make smaller than 500MB and it worked. I think chrome://tracing may be the cause of the problem.

@robogast
Copy link
Author

I was able to analyze the trace through https://ui.perfetto.dev/ (with native acceleration) on chrome (didn't work on firefox)

@botbw
Copy link

botbw commented May 1, 2024

I was able to analyze the trace through https://ui.perfetto.dev/ (with native acceleration) on chrome (didn't work on firefox)

This app works for me ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working plugin PyTorch Profiler TensorBoard Plugin related
Projects
None yet
Development

No branches or pull requests

5 participants