Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add statistics to graph in tensorboard #92

Merged
merged 1 commit into from
Dec 30, 2018
Merged

add statistics to graph in tensorboard #92

merged 1 commit into from
Dec 30, 2018

Conversation

JaCoderX
Copy link
Contributor

collect statistics about computation time and memory usage and present them in tensorboard graph

collect statistics about computation time and memory usage and present them in tensorboard graph
@JaCoderX
Copy link
Contributor Author

JaCoderX commented Dec 29, 2018

I'm in the process of understanding the model performance bottlenecks. so I have added some functionally to record run time statistics for the graph.

base on the tf docs

this solution is only partial for understanding the performance of the model. As it doesn't records the statistics of the entire net.

A better solution would be to place the start point of the statistic gathering at the entry point of the model. so all the model statistics are properly gathered

@Kismuz Kismuz merged commit eeff0f0 into Kismuz:master Dec 30, 2018
@Kismuz
Copy link
Owner

Kismuz commented Dec 30, 2018

LGTM!

@JaCoderX
Copy link
Contributor Author

@Kismuz I think I found a serious flew in my pull request, so I think it is best to revert the commit.

while working on a model I notice that this solution is really memory hungry.
memory seem to grow linearly with the runtime.
plus the tensorboard files get really massive.

just for the protocol, commit was missing this in line 1414:
self.process_summary(sess, data, model_summary, run_metadata=run_metadata)

solution can be optimize to reduce memory usage but I think it just better to revert it

@Kismuz
Copy link
Owner

Kismuz commented Jan 10, 2019

done

Kismuz added a commit that referenced this pull request Jan 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants