Component should support larger attachments

[jhorbulyk]

Other Feature Request

Description

Currently, the component code blocks writing of attachments that are larger than 5MB in size in certain branches and does not appear to enforce limits in other branches. (

xml-component/lib/actions/attachmentToJson.js

Line 8 in b9b1369

const MAX_FILE_SIZE = 5242880; // 5 MiB

) We should apply a constant max size limit to all files. This limit should default to 100MB (100 * 1024 * 1024 bytes) however it should be configurable via a MAX_FILE_SIZE env variable. We should ensure that all Code paths can hande files that are close to this limit.

Definition of Done

• There is a global max attachment size value that is referenced in all actions that write attachments.
• This global max attachment size value should be configurable via component environment variables like the SFTP component. This configuration should use the same units as the SFTP component (MB). When this environment variable is not provided, then this limit should default to 100MB (100 * 1024 * 1024 bytes).
• All non-deprecated actions be able to read & write attachments of up to 100MB under the default memory conditions. (Within densible XML depth.)

[if0s]

@andrewreshitko component can work with up to 10MB files, to support bigger files, we need to change main library and modify actions (ex. attachmentXML-attachmentJSON and backward)

[HannaTrotsenko]

https://github.com/elasticio/xml-component/releases/tag/1.4.0