Failure to Update Gestalt with New Node ID and Issues with Discovery Mechanism Transparency

[CryptoTotalWar]

Describe the Bug(s)

Issue 1: Stale Node ID Data Retention

• The system fails to update gestalts with new Node IDs when a node is destroyed and recreated. Old cryptolinks are not automatically removed, causing outdated information to persist.

Issue 2: Inefficient Discovery Mechanism

• The background discovery system lacks transparency and does not work well with multiple cryptolinks, failing to prioritize the most recent cryptolinks. Manual rediscovery does not update the gestalt with a new Node ID, leading to stale data issues.

To Reproduce

1. User A discovers User B's Node ID with polykey identities discover.
2. User B destroys their node and recreates a new one, leaving the old cryptolink intact.
3. User A attempts to rediscover User B, but the old Node ID remains visible.

Note: For the demo, in order for us to solve for this, manual deletion of the cryptolink on GH was required after deleting the old node.

Expected Behavior

• The discovery mechanism should handle the dynamic nature of nodes and their cryptolinks efficiently.
• A manual rediscovery command should force the system to update the gestalt with new Node IDs.
• A new command should be available to delete old cryptolinks.
• A precedence system for handling cryptolinks should be established, prioritizing the most recent ones.

Proposed Solutions

• Implement an identities status command for progress feedback on discovery tasks.
• Add a feature for manual rediscovery to update Node IDs immediately.
• Create a systematic approach to handling and cleaning up cryptolinks when nodes are destroyed.

Screenshots/Logs

(attach relevant screenshots or logs here, when recreating the issue.)

Platform

• Device: MacBook Air (2021 Mac M1)
• OS: macOS (latest)
• Version: Latest release v0.2.6

Additional Context

• Discussions from Discord

Links to Related GitHub Issues

• Background Discovery Mechanisms: GitHub Issue #691
• identities status Command: GitHub Issue #162
• Vault Sharing With GestaltIDs: GitHub Issue #626

Notify maintainers

(Use git blame or refer to Discord conversations to identify relevant contributors.)

[linear]

ENG-29 Failure to Update Gestalt with New Node ID

[CMCDragonkai]

Looks good. My only recommendations is capturing screenshots of the cryptolinks and a excalidraw diagram that usually explains things that are complex to be more easier to understand. We usually use sequence diagrams. But for this it can be even more basic.

[tegefaulkes]

I think this would normally be two separate Issues, One for each of the problems you outlined at the top.

As for the problems.

Issue 1

The system fails to update gestalts with new Node IDs when a node is destroyed and recreated. Old cryptolinks are not automatically removed, causing outdated information to persist.

What do you mean by re-created? Deleting a node and creating a new one is strictly a new separate node. It doesn't replace the old one implicitly in any form. It is strictly a new node.

Using a recovery code can generate a node from scratch, this can be considered re-creating a node. Functionally it is the same node, same keys and identity. But there is no way for us to re-create the sigchain and by extension any of the cryptiolinks. This means any of the cryptolinks on other nodes or in the github gists are no longer valid for this node. Any links will need to be manually re-created.

So going by the steps outlined above, What you're expecting and seeing as the problem can't actually be implemented. Maybe I'm just missing some details?

Issue 2

The background discovery system lacks transparency and does not work well with multiple cryptolinks, failing to prioritize the most recent cryptolinks. Manual rediscovery does not update the gestalt with a new Node ID, leading to stale data issues.

This seems like a few issues in one?

• Lack of feedback with discovery
• Not working well with multiple cryptolinks? You need to be more clear here. In what way is it not working well? what is it doing, what is expected? Is it the failing to prioritise the most recent cryptolinks? That is also not very clear.
• Manual rediscovery does not update the gestalt with a new Node ID, leading to stale data issues. Are we talking about nodes who had their keys rotated? We've planned to handle nodes who had their IDs change, but that hasn't been implemented yet.

In the future, if there are multiple issues you should break it down to a list and make very clear what each problem is. Make very clear that.

1. What is it doing?
2. What should it be doing?

Also when outlining the reproducible steps, it helps me a lot if you give a very clear list of each command you ran and the order you ran it.

[tegefaulkes]

After discussing it with @CryptoTotalWar I have a clearer idea of the problem. I'll write up some more details later. The core of it is that the discover isn't properly handling multiple claims on a single identity.

[tegefaulkes]

With the recent released version we should be able to properly re-discover a vertex now. @amydevs Also did some testing and found that the discovery process was in fact handling multiple cryptolinks just fine. Automatic removal of dead links is not really something we need right now. If we need that, it can be discussed in #164

So I think all of the main points here are addressed now. I'm going to close this.

[CMCDragonkai]

There is still a problem with a user having multiple cryptolinks on the same GitHub identity. @aryanj recently had this problem where he had claimed GitHub multiple times using the SAME node. And then when I tried to discover github.com:aryanjassal it resulted in a task time out in the agent logs.

It shows up as:

WARN:polykey.PolykeyAgent.task v0phkmdj0tlo0146ppc9ce79g0k:Failed - Reason: ErrorTaskTimeOut

[CMCDragonkai]

Will be starting a new issue on this problem @aryanj unless you've already posted a bug?