How to stream incoming multipart/form-data to 3rd party services using Pez

[RicardoVaranda]

Hey guys,

We are trying to stream multipart/form-data files to a third party storage. We have managed to do this using:

  options: {
    payload: {
      output: 'stream',
      allow: "multipart/form-data",
      parse: false,
    }
  }

The issue with this is that it uploads the files in raw format, while this is ok in some cases for images and videos we would like to keep their type and settings.

Now we could set parse to true which would fix this but it seems the internal parse stores the whole file in memory and before streaming it. This is ok for small files but we deal with lots of file over 1gb so this is not an option.

After a bit of research, we discovered Pez is suitable for this as it parses in parts.

The issue is all of the examples in Pez just store the data locally, how can we stream those parts to a 3rd party?

cc @cjihrig