Seeking guidance on entity reconciliation and managing changes over time

[Ataravia]

Hello everyone,

My name is Nicola, and I'm the founder of ATARAVIA, a company I'm developing to help people with disabilities and other access requirements make informed decisions about travel and participation.

I've been researching the Reconciliation API, OpenRefine, Cocoda, SKOS, and JSKOS to better understand how existing systems identify entities, connect information from different sources, and manage those relationships.

The research has been helpful, but I've reached a point where I think it makes more sense to ask people who have experience working with these systems for direction.

One thing I'm trying to understand is what happens after entities have been matched. If something changes, is merged, or is deleted, how do systems identify what else might be affected? I'm also interested in how people keep track of why a match was originally accepted and what happens when new information calls that decision into question.

I realize some of what I'm exploring may fall outside the scope of the Reconciliation API, and that's perfectly fine. I'm interested in learning what approaches already exist and whether there are other projects, standards, or communities I should be looking at.

I'd appreciate any guidance or suggestions on where to direct my research.

Thank you!

[wetneb]

One thing I'm trying to understand is what happens after entities have been matched. If something changes, is merged, or is deleted, how do systems identify what else might be affected?

There isn't much that is planned in the reconciliation API itself for keeping track of such events. In OpenRefine, users learn that reconciled data can get stale, and that they might need to reconcile again if they wait too much before using it.

If I had to imagine a way to stay in sync, I'd probably try to do it based on existing LOD standards. I don't know if there are good ways to listen to updates of a collection of RDF URIs, apart from fetching the RDF statements about them, one by one.

[Ataravia]

Thank you for responding. That helps me understand where the Reconciliation API's scope ends.

I'll take a closer look at the LOD and RDF approaches you mentioned.

Do you happen to know of any projects or communities looking at how entity relationships are maintained over time, particularly when source entities change or merge?

Thank you again for your time, I really appreciate it!

[wetneb]

I'm only familiar with the Wikidata community, where this problem is partly dealt with by the infrastructure (with redirects and a bot that rewrites entities), but a lot of manual curation work to update links to external databases.

[Ataravia]

Thank you! I'm going to take a closer look at how Wikidata handles redirects, bots, and the manual work involved in keeping relationships current.

This gives me another direction to explore. I appreciate your help!

[acka47]

Do you happen to know of any projects or communities looking at how entity relationships are maintained over time, particularly when source entities change or merge?

Hi @Ataravia , I know about these two projects that have developed approaches for keeping Linked Data entities in sync that should also cover merges:

• Linked Data Event Streams (LDES), a European project: https://interoperable-europe.ec.europa.eu/collection/semic-support-centre/linked-data-event-streams-ldes (See also slides and video from last year's SWIB presentation linked at https://forum.swib.org/t/the-linked-data-event-streams-ldes-profile-for-cultural-heritage-datasets/1450)
• Entity Metadata Management API, an eefort from the mainly US-based [LD4 community](LD4 Community)

I haven't worked yet with neither so I am not the right person to answer questions about details. However, I invite you to join the SWIB (Semantic Web in Libraries) community and forum at https://forum.swib.org/ (I am co-organizer and co.chair of the SWIB cinference's programme committee). The annual conference SWIB26 will happen next week and you are also free to post your questions in the forum at any time. People from both initiatives are there so you should be getting some answers.

[Ataravia]

@acka47 Hi Adrian,

Thank you again for taking the time to respond and for pointing me toward LDES, the Entity Metadata Management API, and SWIB.

I've spent some time exploring the resources you recommended, along with a few related areas. It's been very interesting, particularly the distinction between discovering that information has changed and understanding what that change means for existing relationships and the evidence that depends on them.

That's an important question for what we're building at ATARAVIA, and your recommendations have given me a much better understanding of the work already being done in this area.

I've also registered for the SWIB forum and am waiting for my account to be approved. I'm looking forward to learning more and participating in the discussions.

I really appreciate your willingness to share your knowledge and point me toward these resources.

Nicola