GC or Recovery of deleted entities

[htdvisser]

Summary

Some entities in the IS are soft-delete. We want to be able to clean them up (and release their IDs) or recover them.

Why do we need this?

Sometimes entities are deleted by accident, in which case we want to restore them. After a cooldown period we want to be able to re-register new entities with the same IDs.

What is already there? What do you see now?

On delete, entities are marked with a deleted_at

What is missing? What do you want to see?

• It should be possible to search for or to list entities that have been deleted
• It should be possible to recover deleted entities
• Deleted entities should be permanently deleted after a cooldown period

How do you propose to implement this?

• In ListXXX/GetXXX requests we could add a bool include_deleted
• For restoring deleted entities we can...
  • restore deleted entities on Get/Update
  • restore deleted entities with a new API call
• For cleaning up deleted entities we can...
  • add a background process to the IS that periodically GCs deleted entities
  • add an API endpoint for GCing deleted entities, which can be called from a cron job.

Can you do this yourself and submit a Pull Request?

Sure

[virtualguy]

@htdvisser this would be great to have for re-provisioning devices.

Is there any work around to reuse a dev_eui other than manual manipulation of redis+DB?

[htdvisser]

This particular issue is more about applications and gateways. Sometimes users want to start over and decide to delete their application/gateway, not knowing that the ID will not be released.

Right now the solution is to re-register with a different ID, but sometimes users really like their IDs...

Note that for gateways, the EUIs are released on delete (since #1843).

---

End Devices are not soft-delete, so if you delete those they're gone, and you can re-register them.

The tricky part for End Devices is that they have to be deleted from 4 databases (the Join Server, Application Server, Network Server and Identity Server). The console and command-line interface already do this, but if you have a multi-cluster deployment, and delete your end devices in the "wrong" console, the end devices may still be registered in the other JS/AS/NS. We've also seen users who use the API directly and only delete devices from the Identity Server, again keeping the registrations in JS/AS/NS.

[virtualguy]

Gotcha, makes sense. Sounds like we have a different issue as the NS was complaining of device already existing during create from the CLI tool. I'll try reproduce and open a separate ticket