Skip to content
This repository has been archived by the owner on Jun 24, 2023. It is now read-only.

Migrate to Twitch Webhooks API #3

Open
5 tasks
micalevisk opened this issue May 2, 2020 · 1 comment
Open
5 tasks

Migrate to Twitch Webhooks API #3

micalevisk opened this issue May 2, 2020 · 1 comment
Labels
enhancement New feature or request on hold I'm not working on this

Comments

@micalevisk
Copy link
Owner

micalevisk commented May 2, 2020

To replace the current long polling technique
just subscribe to the topic stream changed and notify when data.length > 0 (from notify request payload) and the date saved on some sort of storage file is not the same as data.started_at (since there is no way to filter only stream goes online event)

This will increase notification reliability, introduces realtime capabilities and decrease the usage of GitHub Actions quota

But GH Actions as storage can no longer be used due to webhook callback constraint (Vercel Now, Netlify, MongoDB Stitch, Supabase Function & storage. And keeping state between serveless function invocations will be a problem. To solve this we can use the free tier of Upstash (Redis-as-a-service), or a Database-as-a-service like: mLab, FaunaDB cloud, Railway, PlanetScale, or even the free tier of Qovery to host the entire Nodejs app + using Redis as the main database.

Maybe we can use External events: repository_dispatch (via GitHub API) to trigger the workflow run that will use an artifact to keep the state. See Async workflows with repository dispatch - GitHub Satellite 2020

GH Actions could be used to renew the webhook subscriptions since "All subscriptions have an expiration time, which cannot exceed 10 days."

Besides that, we'll can implement a notification for multiple Twitch channels and update the message sent by the bot as well 😃

Note that we're looking for a more dependecy-free and slim solution here to reduce the serverless cold start latency -- https://moiva.io to rescue!


The happy path workflow will be somewhat like this:

  • Client: our temporary HTTP client just send a subscription request (besides the fact that subscriptions should be renew due to its expiration time)
  • Helix API: https://api.twitch.tv/helix endpoint
  • Twitch: the twitch webhook server
  • Server: our serverless function that will be used as webhook (hub.callback X value in 1, following this spec)

sequence diagram


TODO to close this issue

  • Choose a serverless host provider for the new serverless/lambda function ~ must be free
    • If the serverless host plataform doesn't comes with some free state management feature, choose one cloud-based (or use GitHub Actions artifacts)
  • Implement the code for the serverless function (to handle GET / and POST / requests from Twitch, and other personal routes for eg: GET /listen/{channel} to subscribe using Helix and Webhook APIs). Including the server communication with the storage mechanism
  • Write the GitHub Action that will renew subscriptions ~ using schedule to run every 10 days since some well defined day (the last run)
  • Add a human-friendly diagram to describe HOW IT WORKS

Example of incoming request from Twitch (step 5):

req url = '/'
req method = 'POST'
req headers = {
  host: 'c56b8fe5.ngrok.io',
  'user-agent': 'Go-http-client/1.1',
  connection: 'close',
  'content-length': '559',
  'content-type': 'application/json; charset=utf-8',
  link: '<https://api.twitch.tv/helix/webhooks/hub>; rel="hub", <https://api.twitch.tv/helix/streams?user_id=433615460>; rel="self"',
  'twitch-notification-id': '88429811-567d-4cfa-85a3-b37b88abe65f',
  'twitch-notification-retry': '0',
  'twitch-notification-timestamp': '2020-05-02T20:03:23Z',
  'x-hub-signature': 'sha256=XYZABC',
  'accept-encoding': 'gzip',
  'x-forwarded-for': '34.218.218.240'
}

{
  data: [
    {
      game_id: '509670',
      id: '1438466417',
      language: 'pt',
      started_at: '2020-05-02T20:00:06Z',
      tag_ids: [Array],
      thumbnail_url: 'https://static-cdn.jtvnw.net/previews-ttv/live_user_codigofalado-{width}x{height}.jpg',
      title: '[BR] Criando um Timer com Vue.js + Novos MEMEs no Canal',
      type: 'live',
      user_id: '433615460',
      user_name: 'CodigoFalado',
      viewer_count: 57
    }
  ]
}
@micalevisk micalevisk added the enhancement New feature or request label May 2, 2020
@micalevisk
Copy link
Owner Author

micalevisk commented May 4, 2020

For now I'm testing the feasibility of this. Soon I'll be prototyping the new design.

@micalevisk micalevisk self-assigned this Oct 7, 2020
@micalevisk micalevisk added the on hold I'm not working on this label Oct 7, 2020
@micalevisk micalevisk pinned this issue Oct 15, 2020
@micalevisk micalevisk added WIP working in progress and removed on hold I'm not working on this labels Oct 18, 2020
@micalevisk micalevisk added on hold I'm not working on this and removed WIP working in progress labels Apr 4, 2021
@micalevisk micalevisk removed their assignment Jul 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request on hold I'm not working on this
Projects
None yet
Development

No branches or pull requests

1 participant