DLM Migrations for Software models

[lampwins]

As a app engineer, I want migrations built in DLM for the porting of the SoftwareLCM and SoftwareImageLCM models into core, so that I can ensure the data is migrated for users.

Create data/schema migrations in support of moving the models into core. Data lives in the models and in custom relationships which will become proper FKs in the new modes. Some features of DLM will need to be refactored to support this move.

Please document this process and lessons learned for future moves of models between apps and core.

[gsnider2195]

@lampwins we discussed this in the design meeting and looks like we might run into some problems with the new models in core.

DLM has a derived many-to-many relationship from Device <-> SoftwareImageLCM through the use of tags but our design uses the Device -> SoftwareVersion -> SoftwareImage to derive that relationship. Here are some use cases we see where that could create a difficult migration and a confusing data model:

Use case 1 - Same version, different images

Existing Data model in DLM

# Device -> SoftwareLCM relationship through a RelationshipAssociation
Device1 -> Version X.y.z
Device2 -> Version X.y.z

# Derived Device -> SoftwareImageLCM relationship using tags
Device1 -> image.x.y.z.bin
Device2 -> image.x.y.z.adventerprise.bin

Data model if migrated to core models as-is

# Device->SoftwareVersion foreign key relationship
Device1 -> Version X.y.z
Device2 -> Version X.y.z AdvEnterprise

# Derived Device -> SoftwareImage relationship through the SoftwareImage->SoftwareVersion FK
Device1 -> Version X.y.z -> image.x.y.z.bin
Device2 -> Version X.y.z AdvEnterprise -> image.x.y.z.adventerprise.bin

• Device -> DeviceType -> SoftwareImage would no longer be a reliable way to filter valid Device -> SoftwareVersion relationships because we would have duplicate SoftwareVersions
• We probably wouldn't be able to automatically determine a suitable version name for the copied SoftwareVersion instances so it would probably end up as Version X.y.z - copy

Use case 2 - Same version, different set of images with overlap

Existing Data model in DLM

# Device -> SoftwareLCM relationship through a RelationshipAssociation
Device1 -> Version A.B.C
Device2 -> Version A.B.C

# Derived Device -> SoftwareImageLCM relationship using tags
Device1 -> system.a.b.c.bin
Device1 -> firmware.foo.bin
Device2 -> system.a.b.c.bin
Device2 -> firmware.bar.bin

Data model if migrated to core models as-is

# Device->SoftwareVersion foreign key relationship
Device1 -> Version A.B.C
Device2 -> Version A.B.C - copy

# Derived Device -> SoftwareImage relationship through the SoftwareImage->SoftwareVersion FK
Device1 -> Version A.B.C -> system.a.b.c.bin
Device1 -> Version A.B.C -> firmware.foo.bin
Device2 -> Version A.B.C - copy -> copy of system.a.b.c.bin
Device2 -> Version A.B.C - copy -> firmware.bar.bin

• In this case we have to make a copy of the SoftwareLCM and SoftwareImageLCM instances
• You would need to be able to set default_image on multiple DeviceType -> SoftwareImage m2m due to duplicate SoftwareImage instances

[gsnider2195]

Ran into this issue today when trying to upgrade the next sandbox

ERRORS:
nautobot_device_lifecycle_mgmt.SoftwareImageLCM.device_types: (fields.E302) Reverse accessor for 'nautobot_device_lifecycle_mgmt.SoftwareImageLCM.device_types' clashes with field name 'dcim.DeviceType.software_images'.
        HINT: Rename field 'dcim.DeviceType.software_images', or add/change a related_name argument to the definition for field 'nautobot_device_lifecycle_mgmt.SoftwareImageLCM.device_types'.
nautobot_device_lifecycle_mgmt.SoftwareImageLCM.device_types: (fields.E303) Reverse query name for 'nautobot_device_lifecycle_mgmt.SoftwareImageLCM.device_types' clashes with field name 'dcim.DeviceType.software_images'.
        HINT: Rename field 'dcim.DeviceType.software_images', or add/change a related_name argument to the definition for field 'nautobot_device_lifecycle_mgmt.SoftwareImageLCM.device_types'.

[lampwins]

Per discussion with the DLM team, the object tags functionality is basically used as an "override" function where a device needs to run some other default image within the version. So we should add the M2M fields to the image, to support such override use cases. We should then be able to migrate the data over, and usage within DLM for this is pretty limited, so if we not able to map everything, that is acceptable.

[gsnider2195]

Will rename SoftwareImage model to SoftwareImageFile

[gsnider2195]

Will also need to add an M2M from SoftwareImageFile to InventoryItem (for feature parity with DLM) and to VirtualMachine for consistency. #5280