Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API that monitors queue status #297

Merged
merged 8 commits into from Feb 23, 2022
Merged

Add API that monitors queue status #297

merged 8 commits into from Feb 23, 2022

Conversation

MrOrz
Copy link
Member

@MrOrz MrOrz commented Feb 19, 2022

Related: TODO items in broken group chat https://g0v.hackmd.io/bhL6csQ8T1e81E2De7ZS5Q#%E7%BE%A4%E7%B5%84%E5%8A%9F%E8%83%BD%E5%A3%9E%E6%8E%89-update

  1. Fixes "Missing process handler for job type __default__" when JOBQUEUE_CONCURRENCY is set #260
  2. Moves queue instance to src/lib/queues.js so that it can be shared across webhook & GraphQL resolvers
  3. Add queue open API that returns queue stats for monitor
    • Dunno know how to test "completed" queue QQ
    • Just added test case for "waiting"

Deployed to staging:
圖片

@MrOrz MrOrz self-assigned this Feb 19, 2022
@MrOrz MrOrz requested a review from nonumpa February 19, 2022 08:09
@MrOrz
Copy link
Member Author

MrOrz commented Feb 21, 2022

On localhost, src/graphql/__tests__/queue.js runs first, and src/webhook/__tests__/index.js silently fails with the following error:

    console.error src/webhook/handlers/groupHandler.js:107
      TypeError: Cannot read properties of undefined (reading 'timestamp')
          at _default.processJob (/Users/johnsonliang/workspace/rumors-line-bot/src/webhook/handlers/groupHandler.js:53:47)
          at processTicksAndRejections (node:internal/process/task_queues:96:5)

In groupHandler:53, it is trying to read job.data.otherFields.timestamp. However, job.data is actually { foo: 'bar' }, the
fixture inserted in returns waiting job info.

On github actions, src/webhook/__tests__/index.js runs first without this error, and there is open handles left after test.

@MrOrz
Copy link
Member Author

MrOrz commented Feb 21, 2022

If src/webhook/__tests__/index.js happened to run before src/graphql/__tests__/queue.js on localhost, there will be snapshot error:

  ● returns waiting job info

    expect(received).toMatchInlineSnapshot(snapshot)

    Snapshot name: `returns waiting job info 1`

    - Snapshot
    + Received

      Object {
        "data": Object {
          "queue": Array [
            Object {
              "jobCounts": Object {
    -           "waiting": 1,
    +           "waiting": 0,
              },
    -         "lastWaitingAt": 1989-06-04T00:00:00.000Z,
    +         "lastWaitingAt": null,
              "queueName": "groupEventQueue",
            },
            Object {
              "jobCounts": Object {
    -           "waiting": 1,
    +           "waiting": 0,
              },
    -         "lastWaitingAt": 1989-06-04T00:00:00.000Z,
    +         "lastWaitingAt": null,
              "queueName": "expiredGroupEventQueue",
            },
          ],
        },
      }

      88 |   MockDate.reset();
      89 |
    > 90 |   expect(resultQueued).toMatchInlineSnapshot(`
         |                        ^
      91 |     Object {
      92 |       "data": Object {
      93 |         "queue": Array [

      at Object.<anonymous> (src/graphql/__tests__/queue.js:90:24)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

Seems that the processor attached in src/webhook/__tests__/index kicks in.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 1883147809

  • 13 of 14 (92.86%) changed or added relevant lines in 5 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.03%) to 87.365%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/graphql/resolvers/QueueInfo.js 8 9 88.89%
Totals Coverage Status
Change from base Build 1337615961: 0.03%
Covered Lines: 980
Relevant Lines: 1107

💛 - Coveralls

Copy link
Member

@nonumpa nonumpa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@MrOrz MrOrz merged commit f01409d into master Feb 23, 2022
@MrOrz MrOrz deleted the queue-monitor branch February 23, 2022 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"Missing process handler for job type __default__" when JOBQUEUE_CONCURRENCY is set
3 participants