Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve logging #5

Open
cohen990 opened this issue Jan 9, 2016 · 1 comment
Open

Improve logging #5

cohen990 opened this issue Jan 9, 2016 · 1 comment

Comments

@cohen990
Copy link
Owner

cohen990 commented Jan 9, 2016

I have reason to believe that as the program processes more and more pages - it slows down.

Need to log more reliable data (instead of arbitrarily declaring where to start the timer and stop the timer, would be more useful to log every time you write to the graph file.

Also need to be able to view the data usefully - some sort of graph scripts - maybe in python

@cohen990
Copy link
Owner Author

After logging around the filewrite - I've found that there are a subset of writes which take upwards of 15,000 ms. It is possible that the flush after every node is the issue - because it has to save the entire file.

❗ Put logging around the flush and see if this is the source of the slowdowns
If it is, batch the flushes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant