[RFE] Support updating existing document

[igarashitm]

Describe the solution you'd like
Currently once you import a document and create mappings, you can't update document schema with keeping existing mappings. When a document schema is updated, e.g. fields are added/removed, you need to start from scratch again. Allow updating documents without discarding existing mappings.
When a field is removed, and that field is in some mappings, we should remove those mappings. Maybe we can show some warning in dialog with showing the reason, "Since field A is removed, corresponding mappings will be removed"

[Shrishti1234]

Hi @igarashitm , I also need this feature as this is very useful. This is pain to start mapping from scratch, if there is any change in source and target document.

[nguyen-vo]

Hi, has anyone had an update on this or has been working on this feature?. I am currently working on this, but when I tried to get the fields from the imported document, they are empty

[pleacu]

So - to be clear - the enhancement is to allow the import of an existing document name (i.e. update the document). The enhancement does not entail dynamically adding fields into the document from the UI. Any missing field in the updated document that is currently mapped will result in a dialog challenge stating that the resulting mapping will be deleted y/n.

i.e. The update of source file myfile.json results in the elimination of a mapped field(s). These mappings will be removed. Proceed?

All other mappings will remain intact. Does that sound correct?

[igarashitm]

That's right. Modifying a field individually has a separate issue - #1417

[igarashitm]

Since this one is going to be not a small logic, we'd be better to implement it at backend side, then UI only delivers new schema and current mappings and receive an updated mappings. It might need a separate REST endpoint to "dry-run" the update and receive those warnings before that, or UI might be able to decide to actually reflect or discard.

[igarashitm]

Note that it's not only add/remove the field, but the field could change COMPLEX<>primitive, changing fieldtype, collection<>non-collection, etc. We can remove related mappings if any of that happens, but at least need to detect all those changes.

[pleacu]

Right - so for the first pass at this - if the updated document removes or changes the type of a mapped field the mapping is removed (post challenge). We can revisit recalculating the mapping with the updated field type in a future enh. I think the lowest hanging fruit is just where a user adds an unrelated field to a document and doesn't want to lose their existing mappings.

Since we have the document management service and we have the docdefs and mappings already established I was thinking that would be a good place to see if we can support this. We'll definitely need an updated REST api once we're ok with updating. wdyt?

[igarashitm]

I would want to avoid having more logic in UI, actually if we have hands I want to move more logic from UI atlasmap-core to backend. That's far better for testing and maintaining than in typescript toolchain.

[igarashitm]

Also, I wonder how it works with the on-demand inspection if the logic is implemented in the UI - what happens if there're fields with the deep path, does it invoke inspections recursively?

[pleacu]

Ok - I'll look pushing as much as possible to the backend. Good point w.r.t. dynamic inspection. It could get hairy.

[igarashitm]

#2214 (comment)

This would need an update icon, as you can import multiple Documents from a same schema. And the "Update Document" icon needs to be next to the "Remove Document" icon.

[igarashitm]

This one turned out to be even more complicated - blocked by the server side field search (#603) in order to handle children fields that's already used in the mappings.

[stale]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[davidg2019]

Hi @igarashitm, will this feature be available soon? We need it to use AtlasMap in our solution. Thank you!

[Shrishti1234]

Hi @davidg2019 ,
There is work around for that If you want to update schema after creating mapping. You can directly change schema in .adm file by renaming it to .zip file. The grammar is wriiten is json and easy to understand.

[davidg2019]

Thanks @Shrishti1234,
I know that and it works even if we do some manipulation to extract and format the schema in a real Json to easily modify it. But my question is still valid, is the feature will be implemented? Thank you

[gnunn1]

+1 on this feature, that workaround is very painful for something that should be an OOTB feature IMHO. If you are dealing with legacy interfaces then yes the source/target documents may not change but when using AtlasMap to iterate on an integration with a new interface/endpoints having to remap everytime the source or target changes makes it impossible to use the tool effectively.