W3name publishing - Use a queueing system which can scale

[adamalton]

Currently all w3name record requests are stored in memory before they are processed by the POST api end point to publish on the DHT. At the current traffic this is perfectly acceptable, but at a certain scale or volume of requests, this might cause the service to fall over.

[joshghent]

@francois-potato to work on a proposal for this work.
Suggested implementation is to use SQS with a FIFO queue.
Using this tool: https://github.com/bbc/sqs-consumer

[mbommerez]

Proposal: https://www.notion.so/ipns-publisher-2-258f425cabf4444b826481918f7b4c1e

[adamalton]

This ticket is really in the wrong project. I've made a replacement ticket in ipns-publish, which is where it belongs: web3-storage/ipns-publisher#8