Measure post queue timings #598
Conversation
Revert "Queue: List/Tuple replace with a Dict for proper functions." This reverts commit 5c23a3f. Revert "Python2/3 Reverse Compat functionality, also 'return' if EOFError" This reverts commit f604590. Revert "Ain't flake great" This reverts commit 13f9301. Revert "Shorten site names" This reverts commit a46f747. Revert "Merge pull request #598 from Charcoal-SE/time-in-queue" This reverts commit ec0b1fa, reversing changes made to 92adc01. Revert "Merge pull request #599 from quartata/master" This reverts commit 92adc01, reversing changes made to 95ee2fe.
|
Just a note that this broke some things, too, so we had to revert it in commit 0177abf, but it's now fixed (via 5c23a3f and subsequent error handling checks in 706420d). (Core issue: dict and lists aren't interchangeable, and we have to do some manipulation to make things work proper as dicts or lists, while also erasing and starting with a fresh pickle (done via 10425ad to forcibly delete queue pickles once at load, which we did by manually failing over to each instance; make sure that 47d0cf9 is also pulled in to keep it from constantly deleting though, @ArtOfCode-, as your instance is the only one that didn't get 47d0cf9's removal of the "Destroy the Pickles" code) |
|
@teward I nuked the bodyfetcher queue manually, because yeah it broke things. |
This should measure how long a post has been in the bodyfetcher queue at the point that the API call for its site is made. It stores the timings locally in
bodyfetcherQueueTimings.pso that stats can be analysed. Also created a Python script to do that analysis.PR'd so I can get a code review, and cc @Undo1 since he was talking about doing this.