-
|
First of all thanks for the outstanding work on this software. I'm trying to use it to mirror a bunch of IRR data from different sources (NTT, RIPE, …). Yet it looks like I have trouble importing RIPE data although it works for NTT data for instance. IRRd version is 4.1.2 and the configuration is the following. ---
irrd:
database_url: 'postgresql://irrd:<redacted>@postgres/irrd'
redis_url: 'redis://:<redacted>@redis:6379/0'
piddir: /var/run/
log:
level: DEBUG
server:
http:
interface: '::0'
port: 8080
whois:
interface: '::0'
max_connections: 50
port: 43
auth:
gnupg_keyring: /opt/gnupg-keyring/
override_password: {hash}
email:
footer: 'Example IRR'
from: irr@eample.com
smtp: localhost
notification_header: |
This is to notify you of changes in the {sources_str} database
or object authorisation failures.
You may receive this message because you are listed in
the notify attribute on the changed object(s), or because
you are listed in the mnt-nfy or upd-to attribute on a maintainer
of the object(s).
rpki:
roa_source: null
roa_import_timer: 3600
pseudo_irr_remarks: |
This AS{asn} route object represents routing data retrieved
from the RPKI. This route object is the result of an automated
RPKI-to-IRR conversion process performed by IRRd.
sources_default:
- NTTCOM
- RIPE
- RIPE-NONAUTH
sources:
NTTCOM:
authoritative: false
import_timer: 86400
import_source:
- ftp://rr1.ntt.net/nttcomRR/nttcom.db.gz
RIPE:
authoritative: false
import_timer: 86400
import_source:
- https://ftp.ripe.net/ripe/dbase/split/ripe.db.as-set.gz
- https://ftp.ripe.net/ripe/dbase/split/ripe.db.aut-num.gz
- https://ftp.ripe.net/ripe/dbase/split/ripe.db.route-set.gz
- https://ftp.ripe.net/ripe/dbase/split/ripe.db.route.gz
- https://ftp.ripe.net/ripe/dbase/split/ripe.db.route6.gz
RIPE-NONAUTH:
authoritative: false
import_timer: 86400
import_source:
- https://ftp.ripe.net/ripe/dbase/ripe-nonauth.db.gzI'm running this in a Docker setup with dedicated redis and Postgres instances. I manage to gather the following logs: It looks like in import the |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 7 replies
-
|
It seems that lines for import_source_serial are missing in those source definitions. |
Beta Was this translation helpful? Give feedback.
-
According to the documentation, they are. |
Beta Was this translation helpful? Give feedback.
-
|
I changed the config for sources so it looks like so: RIPE:
authoritative: false
import_timer: 86400
import_source:
- 'https://ftp.ripe.net/ripe/dbase/split/ripe.db.as-set.gz'
- 'https://ftp.ripe.net/ripe/dbase/split/ripe.db.aut-num.gz'
- 'https://ftp.ripe.net/ripe/dbase/split/ripe.db.route-set.gz'
- 'https://ftp.ripe.net/ripe/dbase/split/ripe.db.route.gz'
- 'https://ftp.ripe.net/ripe/dbase/split/ripe.db.route6.gz'
import_serial_source: 'https://ftp.ripe.net/ripe/dbase/RIPE.CURRENTSERIAL'
RIPE-NONAUTH:
authoritative: false
import_timer: 86400
import_source:
- 'https://ftp.ripe.net/ripe/dbase/ripe-nonauth.db.gz'
import_serial_source: 'https://ftp.ripe.net/ripe/dbase/RIPE-NONAUTH.CURRENTSERIAL'
|
Beta Was this translation helpful? Give feedback.
-
|
RIPE is a large database, give it more time to complete. |
Beta Was this translation helpful? Give feedback.
-
|
In full transparency, I configured RIPE to mirror the full database instead of the split ones. But I used split sources with APNIC and it worked, so I don't think it to be the issue. |
Beta Was this translation helpful? Give feedback.
-
|
Even after waiting for several hours it does not import it. I'm starting to think that it could come from the Docker setup. I'll try the same setup on a VM with recommended settings and I'll report back. |
Beta Was this translation helpful? Give feedback.
-
|
import_serial_source is not mandatory. The documentation is wrong - I'll correct it. I can not tell from the log you shared what else is happening with the RIPE source. Your log spans just over 1 minute, and in that I see nothing unusual. It seems RIPE data is being downloaded, and then the parsing is started (one downloaded file at a time): This can take a while though, RIPE is large. Think 15 minutes to 3 hours depending on your hardware. This is why NRTM is preferred ;) Without more logs, I can't say more. |
Beta Was this translation helpful? Give feedback.
-
|
Trying the exact same config on a VM instead of nesting it in Docker containers worked. Instead of waiting for hours without nothing much being done, the import completed in few minutes. I'll try to dig into the Docker setup to figure out if it's an issue of resources or something else. |
Beta Was this translation helpful? Give feedback.
import_serial_source is not mandatory. The documentation is wrong - I'll correct it.
I can not tell from the log you shared what else is happening with the RIPE source. Your log spans just over 1 minute, and in that I see nothing unusual. It seems RIPE data is being downloaded, and then the parsing is started (one downloaded file at a time):
This can take a while though, RIPE is large. Think 15 minutes to 3 hours depending on your hardware. This is why NRTM is preferred ;)
Without more logs, I can't say more.