IX-F Importer: duplicate address(es) should result in rejection of JSON export and notification of IXP

[ccaputo]

Describe the bug
Have observed this output from the cron run of the Importer:

Ixlan [IXP name] (id=[IXP num])
IX-F url: [IXP URL]
ERROR: get() returned more than one IXFMemberData -- it returned 2!
Traceback (most recent call last):
  File "/srv/www.peeringdb.com/peeringdb_server/management/commands/pdb_ixf_ixp_member_import.py", line 278, in handle
    success = importer.update(
  File "/srv/www.peeringdb.com/peeringdb_server/ixf.py", line 519, in update
    self.parse(data)
  File "/srv/www.peeringdb.com/peeringdb_server/ixf.py", line 808, in parse
    self.parse_members(data.get("member_list", []))
  File "/srv/www.peeringdb.com/peeringdb_server/ixf.py", line 839, in parse_members
    self.parse_connections(
  File "/srv/www.peeringdb.com/peeringdb_server/ixf.py", line 864, in parse_connections
    self.parse_vlans(
  File "/srv/www.peeringdb.com/peeringdb_server/ixf.py", line 1043, in parse_vlans
    ixf_member_data = IXFMemberData.instantiate(
  File "/srv/www.peeringdb.com/peeringdb_server/models.py", line 2926, in instantiate
    instance = cls.objects.get(**id_filters)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/django/db/models/query.py", line 439, in get
    raise self.model.MultipleObjectsReturned(
peeringdb_server.models.IXFMemberData.MultipleObjectsReturned: get() returned more than one IXFMemberData -- it returned 2!

The IXP and the network were automatically notified as follows: (note the duplicated error present in the email)

-----------------------------


Data supplied by your exchange [IXP name] is in conflict with that provided by [network name]. Please work with [network name] to resolve this conflict or correct your data as appropriate. The PeeringDB Admin Committee may assist with this conflict if it is not resolved in a timely manner.


CREATE AS### - #.#.#.40 - IPv6 not set
- speed: 30G
- operational: True
- is_rs_peer: None
- status: ok

-----------------------------

Data supplied by your exchange [IXP name] is in conflict with that provided by [network name]. Please work with [network name] to resolve this conflict or correct your data as appropriate. The PeeringDB Admin Committee may assist with this conflict if it is not resolved in a timely manner.


CREATE AS### - #.#.#.40 - IPv6 not set
- speed: 30G
- operational: True
- is_rs_peer: None
- status: ok

-----------------------------

To Reproduce
Set up an IX-F JSON file with a duplicate IP and try to import it.

Expected behavior
The JSON data should be rejected by the Importer and result in a notification to the IXP. The Importer should not emit a Python stack dump resulting in alert to the Operations Committee.

Who is affected by the problem?
Networks at IXPs with exports of this buggy nature will receive duplicated error reports that may not even be accurate with the intent of the export.

What is the impact?
Wasted effort. Diminishment of the usefulness of the IX-F Importer.

Are there security concerns?
Not that I can think of.

Are there privacy concerns?
Aside from not mentioning the source of the most recent buggy export, not that I can think of.

What is the proposed priority?
Not urgent, but should be fixed.

[grizz]

+1

Any validation errors should notify the IXP, any other errors should continue to notify ops.

[arnoldnipper]

Any validation errors should notify the IXP, any other errors should continue to notify ops.

From an AC pov I would like us to try harder (i.e. try to fix as many errors as possible. E.g. in this case there was an identical entry. Hence, could have been resolved easily). From a PC pov I can understand that we reject an import as soon as our importer runs into an error.

@peeringdb/ac, what is your opinion?

[peterhelmenstine]

If there are clearly laid out steps to help resolve the conflicts then we (AC) could work on them, otherwise error out as a conflict and notify with an explanation.

[arnoldnipper]

Routed through to 3a