Support for pushing data to external services

[SCdF]

Basic support for pushing arbitrary JSON data to a REST endpoint based
on data in a data_record. Support for Muso SIH's authentication method
as well as Basic Auth.

Documentation: https://github.com/medic/medic-docs/blob/master/configuration/outbound.md

#5499

[kennsippell]

My feedback is mostly about variable naming and interfaces.

[SCdF]

@kennsippell I also added this last commit: bb6dc19 (and me fixing the test: f94a155 )

Which extracts out the singlePush function as discussed, but also adding a todo that I forgot earlier, which is adding a mark to the -info doc on success.

[kennsippell]

Thanks for updating all the nit-picky stuff... I think this is getting close!

Would you also resolve merge conflicts for the next one?

[kennsippell]

It's obviously a bit late for this, but I did want to make a note on some architecture feedback for this PR. I wasn't involved in the design of this feature - so I'm coming at this with hindsight only and perhaps an slightly incomplete understanding of the requirements. It's likely not actionable at this point, but just tossing my hat into the ring for how we evolve this feature over time.

The first thing we're making here is a pretty simple queuing system which uses CouchDB docs to track objects in the queue. There are already some comments about potential optimisations [1] [2]. And some questions about next steps in review -- removing items after they fail x times, alerting about failed items, etc. imo -- queues are easy to build a first version, but costly to scale and perfect. Instead of pushing a doc to into CouchDB, this use-case seems like a good candidate for systems like AWS SQS or alike. Low-cost, tested, high-scale, parallel ready. imo by re-using instead of re-building, we could get more by building less.

The second is our new extensible system for transforming documents into the format expected by whatever service we integrate with. Extensibility is awesome! But instead of provided a single core transition which we hope will fit all integrations, we could think about have partners be able to write their own plug-ins or sentinel transitions. We have seen SIH-specific requirements creep into this first iteration already. I just think it would be a more robust tool to provide to partners which is less code and less platform for us to maintain. If the above SQS suggestion is liked, AWS Lambda could be an excellent mechanism to allow partners to process their queue.