Skip to content
Browse files

Make the dmoz test thing do 30 URLs at once in parallel.

  • Loading branch information...
1 parent e106057 commit a84f455c15ecdbc76f6734df9643e97f14bd9eec @bitplane committed Feb 26, 2012
Showing with 12 additions and 0 deletions.
  1. +12 −0 downloader/testDmoz.sh
View
12 downloader/testDmoz.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+
+# Pick a random sample of 30 URLs and test them. This
+# will give us some idea of how clean the data is and
+# how good the detector is.
+
+for i in `cat ./inputdata/dmoz.txt | shuf | head -30`
+do
+ echo $i `./testSite.sh $i` &
+done
+
+wait

0 comments on commit a84f455

Please sign in to comment.
Something went wrong with that request. Please try again.