Migrate External Sync Units to file upload and add confirmation skip

[gasperzgonec]

Description

These changes change the behavior of external sync unit upload.
This, instead of sending data inside of the event, stores the external sync units inside of an artifact that's uploaded and we only send the artifact list.

This change is done entirely behind the scenes, so no end-user changes are required.

Current batch size is set to 25000.

Connected Issues

• #ISS-265202

Checklist

• Tests added/updated and ran with npm run test OR no tests needed.
• Ran backwards compatibility tests with npm run test:backwards-compatibility.
• Code formatted and checked with npm run lint.
• Tested airdrop-template linked to this PR.
• Documentation updated and provided a link to PR / new docs OR no docs needed.

[Aakkash-Suresh]

Does this mean, all external sync units discovery are now propagated through Artifacts?

[gasperzgonec]

That is correct, yes.

[radovanjorgic]

Added few comments. I am curious if the way how external sync units worker is written will change? Are we still going to support only passing external sync units array in event data and upload it as an artifact in SDK, or we plan to tell developers to start using repos for this? Also, we are ignoring event data length here compeltely (the logic that you have for tracking event data length and emitting if it is over 80%), can we do anything about it?

[gasperzgonec]

Added few comments. I am curious if the way how external sync units worker is written will change? Are we still going to support only passing external sync units array in event data and upload it as an artifact in SDK, or we plan to tell developers to start using repos for this? Also, we are ignoring event data length here compeltely (the logic that you have for tracking event data length and emitting if it is over 80%), can we do anything about it?

We aren't ignoring the event data length here. It's just different. Currently, our issue with event data length was due to the content of the message being sent to SQS being too large. Now we only send the list of files, which is this case will always be maximum of 5 items (the max limit is 125k and we have max 5 batches of 25k).

I would deprecate the current way, and migrate users to the new way of pushing to repo, but I wasn't sure on how to deprecate this.

[radovanjorgic]

LG, maybe you could use that AirSyncDefaultItemTypes enum instead of that constant wherever needed in project.