Relations import, isReferencing is dropped in the target site after being filled by import_content?

[fredvd]

Something funny is going on with the relations import.

• I think I forgot to import relations in my target Plone 5.2 site. When I use @@inspect_relations from collective.relationhelpers I see around 1800 isReferencing relations in the target site.
• So I exported relations again from the source Plone 4 site, import them in the target site.
• Now my 1800 isReferencing relations are gone, but 800 relatesTo (related items) relations have been added.

In import_other isReferencing is added to the ignores list:

collective.exportimport/src/collective/exportimport/import_other.py

Lines 271 to 274 in 5270f95

	ignore = [
	"translationOf", # old LinguaPlone
	"isReferencing", # linkintegrity
	"internal_references", # obsolete

From the rest of the code in this method it seems all existing relations in the target site are removed and only the 'sanitised' relations are imported again. but this destroys all linkintegrity relations.

I assume the import_content imports are recreating the isReferencing relations while restoring the content items.

The most obvious fix would be to not drop the relations

in:

collective.exportimport/src/collective/exportimport/import_other.py

Lines 287 to 297 in 5270f95

	for rel in data:
	if rel["relationship"] in ignore:
	continue
	rel["from_attribute"] = self.get_from_attribute(rel)
	all_fixed_relations.append(rel)
	all_fixed_relations = sorted(
	all_fixed_relations, key=itemgetter("from_uuid", "from_attribute")
	)
	relapi.purge_relations()
	relapi.cleanup_intids()
	relapi.restore_relations(all_relations=all_fixed_relations)

But this does not work, I get ObjectMissing Errors in z3c.relationfield.event.updateRelations where it tries to list existing relations:

[9] > /Users/fred/.buildout/eggs/cp38/z3c.relationfield-0.9.0-py3.8.egg/z3c/relationfield/event.py(81)updateRelations()
-> rels = list(catalog.findRelations({'from_id': obj_id}))
[10]   /Users/fred/.buildout/eggs/cp38/zc.relation-1.1.post2-py3.8.egg/zc/relation/catalog.py(734)<genexpr>()
-> return (self._relTools['load'](t, self, cache) for t in tokens)
[11]   /Users/fred/.buildout/eggs/cp38/z3c.relationfield-0.9.0-py3.8.egg/z3c/relationfield/index.py(49)load()
-> return intids.getObject(token)
[12]   /Users/fred/.buildout/eggs/cp38/zope.intid-4.3.0-py3.8.egg/zope/intid/__init__.py(89)getObject()
-> raise ObjectMissingError(id)

Stranger is that when I step back in the debugger to frame 9 and execute the same line again I do get the rels list:

(Pdb++) list(catalog.findRelations({'from_id': obj_id}))
[<z3c.relationfield.relation.RelationValue object at 0x11eefdac0 oid 0x200b0 in <Connection at 10d0bb760>>]

Or I could remove isReferencing from the ignores list

but then I'm importing the Plone 4 Archetypes linkintegrity lists on recreated Dexterity Content. It could work as no paths/ids etc have changed and for non existing content the relations will be dropped. But it doesn't feel very clean to do this.

[edit:] I tried this and the isReferencing is actually increasing when I include isReferencing relations on import. So importing content creates 1848 isReferencing relatins, then restoing relations ups isReferencing to 1941 relations. :-O

Export existing relations on the target site and merge them with the imported relations.

Meh. I mean doing this in the import_code: make an extra import on the fly, merge with the incoming relations.json and reapply, dropping isReferencing only from the relations.json .

@pbauer How did you deal with this so far while using collective.exportimport?