Setting the memory limit isn't needed. At this moment you setting it lower. By default it is WP_MAX_MEMORY_LIMIT and that is 256M.
I was seeing memory limit errors while developing it, I don't think the defined max limit was actually being applied. Would need to test this again.
Fixed - thanks!