replaced sdwdate bash with sdwdate python rewrite by @troubadoour

Merge remote-tracking branch 'troubadoour/python' Conflicts: usr/lib/sdwdate/modules.d/sdwdate usr/share/test_d_files/40_pool_one_invalid
Kicksecure · Nov 20, 2015 · 7071842 · 7071842
2 parents 4f6f532 + fa1d317
commit 7071842
Show file tree

Hide file tree

Showing 16 changed files with 1,489 additions and 1,963 deletions.
diff --git a/debian/control b/debian/control
@@ -16,8 +16,9 @@ Package: sdwdate
 Architecture: all
 Depends: sudo, logrotate, bc,
  ruby | ruby-interpreter, ruby-inline, ruby-dev,
- ruby-all-dev, gcc, libc6-dev, python-dateutil,
- python-pysocks | python-socksipy, python, tor, ${misc:Depends}
+ ruby-all-dev, gcc, libc6-dev, python-dateutil, python-gevent,
+ python-pysocks | python-socksipy, python-guimessages, python, tor,
+ ${misc:Depends}
 Recommends: vbox-disable-timesync, timesanitycheck, bootclockrandomization,
  timesync
 Suggests: sdwdate-plugin-anon-shared-con-check,

diff --git a/etc/apparmor.d/usr.lib.sdwdate.url_to_unixtime b/etc/apparmor.d/usr.lib.sdwdate.url_to_unixtime
@@ -0,0 +1,10 @@
+# Last Modified: Mon Sep 14 11:33:39 2015
+#include <tunables/global>
+
+/usr/lib/sdwdate/url_to_unixtime {
+  #include <abstractions/base>
+  #include <abstractions/python>
+
+  /usr/lib/sdwdate/url_to_unixtime r,
+
+}
diff --git a/etc/sdwdate.d/30_default.conf b/etc/sdwdate.d/30_default.conf
@@ -0,0 +1,134 @@
+## This file is part of Whonix.
+## Copyright (C) 2012 - 2014 Patrick Schleizer <adrelanos@riseup.net>
+## See the file COPYING for copying conditions.
+
+## Please use "/etc/sdwdate.d/50_user.conf" for your custom
+## configuration, which will override the defaults found here.
+## When sdwdate is updated, this file may be overwritten.
+
+## Proxy settings for non anonymous distributions.
+## Uncomment for standard tor configuration (no stream isolation).
+#PROXY_IP=127.0.0.1
+#PROXY_PORT=9050
+
+## Allowed percentage of url failures common to every pool.
+## If sdwdate frequently stops with "Maximum allowed number of failures" error,
+## create a file "/etc/sdwdate.d/50_user.conf" overriding MAX_FAILURE_RATIO
+## with a higher figure.
+MAX_FAILURE_RATIO=0.34
+
+## pool syntax
+## "url.onion[:port]#comment"
+## "
+##    url.onion[:port]#comment
+##    [url.onion[:port]#comment]
+##    [url.onion[:port]#comment]
+##    [...]
+## "
+## "url.onion[:port]#comment"
+## ...
+
+## pool one.
+##    SecureDrop List
+##       info:
+##          Last updated Thu Oct 23 16:15:00 PDT 2014
+##          Organization	Landing Page	Tor Hidden Service Address
+##       in use:
+##          https://freedom.press/securedrop/directory
+##          https://freedom.press/sites/default/files/securedrop_list.txt
+##          https://freedom.press/sites/default/files/securedrop_list.txt.asc
+##          https://freedom.press/sites/default/files/securedrop.asc
+##       removed because down:
+##          "bczjr6ciiblco5ti.onion#Forbes	https://safesource.forbes.com	bczjr6ciiblco5ti.onion"
+##          "l7rt5kabupal7eo7.onion#BayLeaks	https://bayleaks.com	l7rt5kabupal7eo7.onion"
+##   individual websites
+SDWDATE_POOL_ONE=(
+         "dtsxnd3ykn32ywv6.onion#BalkanLeaks	https://www.balkanleaks.eu	dtsxnd3ykn32ywv6.onion"
+         "znig4bc5rlwyj4mz.onion#ExposeFacts	https://exposefacts.org	znig4bc5rlwyj4mz.onion"
+         "vtjkwwcq5osuo6uq.onion#Greenpeace New Zealand	https://www.safesource.org.nz	vtjkwwcq5osuo6uq.onion"
+         "33y6fjyhs3phzfjj.onion#The Guardian	https://securedrop.theguardian.com	33y6fjyhs3phzfjj.onion"
+         "y6xjgkgwj47us5ca.onion#The Intercept	https://firstlook.org/theintercept/securedrop	y6xjgkgwj47us5ca.onion"
+         "strngbxhwyuu37a3.onion#The New Yorker	https://projects.newyorker.com/strongbox	strngbxhwyuu37a3.onion"
+         "swdi5ymnwmrqhycl.onion#NRKbeta	https://nrkbeta.no/tips	swdi5ymnwmrqhycl.onion"
+         "dqeasamlf3jld2kz.onion#Project On Gov't Oversight (POGO)	https://securedrop.pogo.org	dqeasamlf3jld2kz.onion"
+         "pubdrop4dw6rk3aq.onion#ProPublica	https://securedrop.propublica.org	pubdrop4dw6rk3aq.onion"
+         "hkjpnjbvhrxjvikd.onion#Radio24syv	https://securedrop.radio24syv.dk	hkjpnjbvhrxjvikd.onion"
+         "v6gdwmm7ed4oifvd.onion#Barton Gellman	https://tcfmailvault.info	v6gdwmm7ed4oifvd.onion"
+         "vbmwh445kf3fs2v4.onion#The Washington Post	https://ssl.washingtonpost.com/securedrop	vbmwh445kf3fs2v4.onion"
+         "poulsensqiv6ocq4.onion#Wired's Kevin Poulsen	https://pressfreedomfoundation.org/about/tech/kevin-poulsen	poulsensqiv6ocq4.onion"
+         "tigas3l7uusztiqu.onion#https://mike.tig.as tinkerer at ProPublica in New York"
+)
+
+## pool two.
+##   Hosted by Thomas White List
+##
+##   GlobalLeaks List
+##     info:
+##       https://en.wikipedia.org/wiki/GlobaLeaks#Implementations
+##       http://www.webcitation.org/6WBrtPlrq
+##       Name of organization 	Implementation date 	Category 	Tor Url 	Tor2web Url 	Country
+##     removed because down:
+##       Perun[23] 	2012-April-7 	Investigative Journalism 	Closed 	Closed 	Serbia
+##       "jeuhrnvdyr3xyqz3.onion#Internet Governance Transparency Initiative 	2014-April-5 	Transparency Activism 	jeuhrnvdyr3xyqz3.onion 	https://jeuhrnvdyr3xyqz3.tor2web.org 	Unknown"
+##       "ea433ils4wtprqbv.onion#EcuadorTransparente 	2014-June-19 	Transparency Activism 	ea433ils4wtprqbv.onion 	https://ea433ils4wtprqbv.tor2web.org/ 	Ecuador"
+##       "3qnry3qqjvc2u3c4.onion#ManxLeaks 	2014-July-07 	Transparency Activism 	3qnry3qqjvc2u3c4.onion 	https://3qnry3qqjvc2u3c4.tor2web.org 	Isle of Man"
+SDWDATE_POOL_TWO=(
+         [
+            "atlas777hhh7mcs7.onion#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html"
+            "compass6vpxj32p3.onion#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html"
+            "globe223ezvh6bps.onion#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html"
+            "bbbbbb6qtmqg65g6.onion#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html"
+            "pppppptkftqqnfsq.onion#Hosted by Thomas White. https://www.whonix.org/pipermail/whonix-devel/2015-February/000297.html"
+         ]
+         "w6csjytbrl273che.onion#Ljost[24][25] 	2012-September-30 	Transparency Activism 	w6csjytbrl273che.onion 	https://w6csjytbrl273che.tor2web.org/ 	Iceland"
+         "ak2uqfavwgmjrvtu.onion#MagyarLeaks[26] 	2013-July-7 	Investigative Journalism 	ak2uqfavwgmjrvtu.onion 	https://ak2uqfavwgmjrvtu.tor2web.org 	Hungary"
+         "yn6ocmvu4ok3k3al.onion#Publeaks [27][28] 	2013-September-9 	+40 National/Local Media Consortium 	yn6ocmvu4ok3k3al.onion 	https://secure.publeaks.nl 	Netherlands"
+         "acabtd4btrxjjrvr.onion#Pistajka 	2013-September 	Anticorruption activism 	acabtd4btrxjjrvr.onion 	https://acabtd4btrxjjrvr.tor2web.org 	Serbia"
+         "5r4bjnjug3apqdii.onion#Irpileaks[29][30] 	2013-October-7 	Investigative Journalism 	5r4bjnjug3apqdii.onion 	https://5r4bjnjug3apqdii.tor2web.org/ 	Italy"
+         "2dermafialks7aai.onion#Mafialeaks [31][32][33] 	2013-November-5 	Anti Mafia Activism 	2dermafialks7aai.onion 	https://secure.mafialeaks.org 	Italy"
+         "ymi7h25hgp3bj63v.onion#InfodioLeaks 	2014-January-28 	Anticorruption Activism 	ymi7h25hgp3bj63v.onion 	https://ymi7h25hgp3bj63v.tor2web.org 	Venezuela"
+         "ppdz5djzpo3w5k2z.onion#WildLeaks [34][35][36][37][38][39] 	2014-February-7 	WildLife Crime Activism 	ppdz5djzpo3w5k2z.onion 	https://secure.wildleaks.org 	United States/Africa"
+         "pltloztihmfrg2sw.onion#Salzburger-Piratenpartei 	2014-March-4 	Activism 	pltloztihmfrg2sw.onion 	https://pltloztihmfrg2sw.tor2web.org 	Austria"
+         "ur5b2b4brz427ygh.onion#Nawaatleaks [40] 	2014-March-27 	Activism 	ur5b2b4brz427ygh.onion 	https://ur5b2b4brz427ygh.tor2web.org 	Tunisia"
+         "w6csjytbrl273che.onion#Filtrala [41][42] 	2014-April-23 	Anticorruption Activism 	w6csjytbrl273che.onion 	https://w6csjytbrl273che.tor2web.org/ 	Spain"
+         "abkjckdgoabr7bmm.onion#MediaDirect [43] 	2014-May-11 	Transparency Activism 	abkjckdgoabr7bmm.onion 	https://abkjckdgoabr7bmm.tor2web.org 	Australia"
+         "5r4bjnjug3apqdii.onion#ExpoLeaks[44] [45] [46] 	2014-June-10 	Investigative Journalism 	5r4bjnjug3apqdii.onion 	https://5r4bjnjug3apqdii.tor2web.org/ 	Italy"
+         "bqs3dobnazs7h4u4.onion#ExtremeLeaks 	2014-June-18 	Investigative Journalism 	bqs3dobnazs7h4u4.onion 	https://www.extremeleaks.org/ 	Norway"
+         "fkut2p37apcg6l7f.onion#Allerta Anticorruzione[47][48] 	2014-October-14 	Anticorruption Activism 	fkut2p37apcg6l7f.onion 	https://alac.transparency.it 	Italy"
+         "6iolddfbfinntq2b.onion#Brussels Leaks 	2014-October 24 	Europe Focus Anticorruption Transparency Activism 	6iolddfbfinntq2b.onion 	https://6iolddfbfinntq2b.tor2web.org 	Belgium"
+)
+
+## pool three.
+##   info:
+##     individual websites
+##     riseup.net List
+##        https://help.riseup.net/en/tor#riseups-tor-hidden-services
+##   removed because down:
+##     "suw74isz7wqzpmgu.onion:80#https://www.wikileaks.org/wiki/WikiLeaks:Tor"
+##   removed because no http:
+##     "4cjw6cwpeaeppfqz.onion#xmpp.riseup.net:  4cjw6cwpeaeppfqz.onion (ports 5222, 5269)"
+SDWDATE_POOL_THREE=(
+         "3g2upl4pq6kufc4m.onion#https://duck.co/forum/thread/1762/is-the-duckduckgo-hidden-service-legitimate"
+         "dju2peblv7upfz3q.onion#https://guardianproject.info/2014/10/16/reducing-metadata-leakage-from-software-updates/"
+         "msydqstlz2kzerdg.onion#https://ahmia.fi/address/msydqstlz2kzerdg"
+         "uj3wazyk5u4hnvtk.onion#https://thepiratebay.se/blog/238"
+         "bitmailendavkbec.onion#https://bitmessage.org/forum/index.php?topic=1556.0"
+         "wi7qkxyrdpu5cmvr.onion#Austici www.autistici.org/en/stuff/man_anon/tor.html"
+         "ic6au7wa3f6naxjq.onion#https://lists.gnupg.org/pipermail/gnupg-users/2014-April/049578.html"
+         [
+            "nzh3fv6jc6jskki3.onion#help.riseup.net:  nzh3fv6jc6jskki3.onion (port 443)"
+            "cwoiopiifrlzcuos.onion#black.riseup.net: cwoiopiifrlzcuos.onion (port 443)"
+            "zsolxunfmbfuq7wf.onion#imap.riseup.net:  zsolxunfmbfuq7wf.onion (port 993)"
+            "yfm6sdhnfbulplsw.onion#labs.riseup.net:  yfm6sdhnfbulplsw.onion (port 80, 443)"
+            "xpgylzydxykgdqyg.onion#lists.riseup.net: xpgylzydxykgdqyg.onion (port 80, 443)"
+            "zsolxunfmbfuq7wf.onion#mail.riseup.net:  zsolxunfmbfuq7wf.onion (ports 443, 465, 587)"
+            "5jp7xtmox6jyoqd5.onion#pad.riseup.net:   5jp7xtmox6jyoqd5.onion (port 443) (note: only works with https://5jp7xtmox6jyoqd5.onion)"
+            "zsolxunfmbfuq7wf.onion#pop.riseup.net:   zsolxunfmbfuq7wf.onion (port 995)"
+            "zsolxunfmbfuq7wf.onion#smtp.riseup.net:  zsolxunfmbfuq7wf.onion (ports 465, 587)"
+            "j6uhdvbhz74oefxf.onion#user.riseup.net:  j6uhdvbhz74oefxf.onion (port 80, 443)"
+            "7lvd7fa5yfbdqaii.onion#we.riseup.net:    7lvd7fa5yfbdqaii.onion (port 443)"
+         ]
+         "timaq4ygg2iegci7.onion#https://github.com/meejah/txtorcon http://txtorcon.readthedocs.org"
+         "344c6kbnjnljjzlz.onion#VFEmail https://www.vfemail.net"
+         "fncuwbiisyh6ak3i.onion#https://keybase.io/docs/command_line/tor"
+)