Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
asn_get_routeviews broken #137
Comments
poeml
added
bug
resolved
labels
Jun 5, 2015
poeml
closed this
Jun 5, 2015
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
poeml commentedJun 5, 2015
Issue migrated (2015-06-05) from old issue tracker http://mirrorbrain.org/issues/issue137
msg476 (view) Author: floeff Date: 2014-01-27.09:40:26
It seems asn_get_routeviews | asn_import -b main is broken:
Downloading http://mirrorbrain.org/routeviews/oix-full-snapshot-latest.dat.bz2
Traceback (most recent call last):
File "/usr/bin/asn_get_routeviews", line 197, in
main()
File "/usr/bin/asn_get_routeviews", line 188, in main
for pfx, asnb, asn in pfxasn_uniq:
File "/usr/bin/asn_get_routeviews", line 132, in gen_firstuniq
for tupl in tupls:
File "/usr/bin/asn_get_routeviews", line 100, in gen_asn
for line in lines:
File "/usr/bin/asn_get_routeviews", line 82, in gen_grep
for line in lines:
File "/usr/bin/asn_get_routeviews", line 66, in gen_lines
for line in lines:
File "/usr/bin/asn_get_routeviews", line 48, in gen_cat
for item in s:
EOFError: compressed file ended before the logical end-of-stream was detected
This also leads to the following error message for mb update -a:
psycopg2.IntegrityError: null value in column "prefix" violates not-null constraint
Probably it's related to those messages:
mirrorservice.org: updating network prefix (212.219.0.0/16 -> None)
ftp.rezopole.net: updating network prefix (77.95.64.0/22 -> None)
nexcess.net: updating network prefix (208.69.120.0/21 -> None)
free.nchc.org.tw: updating network prefix (211.73.64.0/19 -> None)
ftp.novell.hu: updating network prefix (213.163.0.0/18 -> None)
etc.
msg478 (view) Author: poeml Date: 2014-01-27.20:34:30
The file is indeed broken.
bzcat oix-full-snapshot-latest.dat.bz2 > /dev/null
bzcat: Compressed file ends unexpectedly;
perhaps it is corrupted? Possible reason follows.
[...]
msg479 (view) Author: poeml Date: 2014-01-27.20:55:15
But the routing table snapshots are provided every few hours, and apparently the issue has been fixed. The
version of the file currently mirrored on mirrorbrain.org works.
Let me know if you still get database errors after re-running the command!
Peter
msg481 (view) Author: floeff Date: 2014-01-27.21:03:17
Partially indeed - the download works again. However, mb main update yields
errors still. Is this a bug in the file, or something different?
mb -b main update --all --all-mirrors
ftp.rezopole.net: updating network prefix (77.95.64.0/22 -> None)
Traceback (most recent call last):
File "/usr/bin/mb", line 1655, in
r = mirrordoctor.main()
File "/usr/lib/pymodules/python2.7/cmdln.py", line 257, in main
return self.cmd(args)
File "/usr/lib/pymodules/python2.7/cmdln.py", line 280, in cmd
retval = self.onecmd(argv)
File "/usr/lib/pymodules/python2.7/cmdln.py", line 412, in onecmd
return self._dispatch_cmd(handler, argv)
File "/usr/lib/pymodules/python2.7/cmdln.py", line 1100, in _dispatch_cmd
return handler(argv[0], opts, *args)
File "/usr/bin/mb", line 439, in do_update
mirror.prefix = res.prefix
File "", line 1, in
File "/usr/lib/pymodules/python2.7/sqlobject/main.py", line 1049, in _SO_setValue
dbValue)])
File "/usr/lib/pymodules/python2.7/sqlobject/dbconnection.py", line 523, in
_SO_update
self.sqlrepr(so.id)))
File "/usr/lib/pymodules/python2.7/sqlobject/dbconnection.py", line 349, in query
return self._runWithConnection(self._query, s)
File "/usr/lib/pymodules/python2.7/sqlobject/dbconnection.py", line 262, in
_runWithConnection
val = meth(conn, *args)
File "/usr/lib/pymodules/python2.7/sqlobject/dbconnection.py", line 346, in _query
self._executeRetry(conn, conn.cursor(), s)
File "/usr/lib/pymodules/python2.7/sqlobject/dbconnection.py", line 341, in
_executeRetry
return cursor.execute(query)
psycopg2.IntegrityError: null value in column "prefix" violates not-null constraint
msg484 (view) Author: poeml Date: 2014-01-27.21:24:16
Apparently, there is no network prefix in the current routing data snapshot which
contains 77.95.64.41. Too bad, shouldn't happen I guess. It's quite strange and would
mean that the routing data snapshot is incomplete. But, it will probably fix itself
automatically...
I can reproduce the error here for another mirror: ftp.piotrkosoft.net
As for the error spit out by "mb update", I have a local fix here that I'll commit to
SVN now, which makes mb cope like this:
mb update --all ftp.piotrkosoft.net
ftp.piotrkosoft.net: STRANGE! There's no prefix containing this hosts IP address
(178.217.221.233)...
ftp.piotrkosoft.net: STRANGE! There's no ASN containing this hosts IP address
(178.217.221.233)...
ftp.piotrkosoft.net: updating geographical coordinates (50.033 19.233 -> 52.0 20.0)
So the command completes to run.
msg485 (view) Author: poeml Date: 2014-01-27.21:31:39
the fix is here:
http://svn.mirrorbrain.org/viewvc/mirrorbrain/trunk/mb/mb.py?r1=8342&r2=8343
(end of migrated issue)