Conspiracy - The Inside Job Runner

[TheConnMan]

Stateless, AWS backed job proxy.

Business Case

SaaS companies often have a publicly accessible API and a number of internal, private APIs for internal services. These services might have endpoints for running cleanup jobs, system remediation, service scaling, or other operational tasks. Monitoring to discover issues which need these API, however, might be provided by a third party. This means notifications from a third party need to securely get inside the VPN to hit the private APIs.

Conspiracy is a job runner which uses AWS as a secure exchange between a third party and internal APIs. The third party sends a job request through SQS which Conspiracy, running inside your VPN, pulls the job and hits an internal API. Conspiracy is purposefully kept simple so it stays a small job proxy; all logic of the job should be behind the internal API.

MVP

• Pull SQS jobs
• Jobs include:
  • URL
  • HTTP method
  • Params object
  • Body for POST/PUT
• Basic logging

Additional Features

• URL template with env var resolution
• "http://job.{{env}}.theconnman.com", ENV=dev
• StatsD
• Dead letter queue
• Timeouts

[TheConnMan]

This now exists at https://github.com/TheConnMan/conspiracy