You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have reason to believe that as the program processes more and more pages - it slows down.
Need to log more reliable data (instead of arbitrarily declaring where to start the timer and stop the timer, would be more useful to log every time you write to the graph file.
Also need to be able to view the data usefully - some sort of graph scripts - maybe in python
The text was updated successfully, but these errors were encountered:
After logging around the filewrite - I've found that there are a subset of writes which take upwards of 15,000 ms. It is possible that the flush after every node is the issue - because it has to save the entire file.
❗ Put logging around the flush and see if this is the source of the slowdowns
If it is, batch the flushes.
I have reason to believe that as the program processes more and more pages - it slows down.
Need to log more reliable data (instead of arbitrarily declaring where to start the timer and stop the timer, would be more useful to log every time you write to the graph file.
Also need to be able to view the data usefully - some sort of graph scripts - maybe in python
The text was updated successfully, but these errors were encountered: