Permalink
Browse files

Fix excessive looping on queued data

Let's say you need to catch up blocks for a few days, on completion
abe_loader will keep trying to load every single block that came in
between, causing excessive looping even once all blocks got imported.

This patch flushes all lines after returning from Abe's load run, yet in
ensures any block that came in while loading won't be missed (at the
possible cost of doing just one extra load).

Timeout also been increased to 15 minutes. No point in trying every 30
seconds, and this could hide an actual issue with the block regex.
  • Loading branch information...
dermoth committed Aug 20, 2016
1 parent b971557 commit 9cd6ed69f89a0e32dfa8bf32873261c203a9cd0b
Showing with 8 additions and 1 deletion.
  1. +8 −1 tools/abe_loader
View
@@ -5,7 +5,7 @@
BLOCK_ACCEPTED="UpdateTip: new best="
# XXX Should be an option. Maximum number of seconds between runs.
-MEMPOOL_INTERVAL=30
+MEMPOOL_INTERVAL=900
usage() {
cat <<EOF
@@ -45,5 +45,12 @@ grep -F --line-buffered -- "$BLOCK_ACCEPTED" |
while true
do
python -m Abe.abe "$@" --no-serve < /dev/null
+ # Flush all input here so that buffered data doesn't make us loop excessively
+ gosleep=1
+ while read -t 1 a
+ do
+ gosleep=0
+ done
+ ((gosleep)) || continue
read -t "$MEMPOOL_INTERVAL" a
done

0 comments on commit 9cd6ed6

Please sign in to comment.