Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure cron scheduler uses the new WorkOrders module #1161

Closed
stuartc opened this issue Sep 28, 2023 · 2 comments
Closed

Ensure cron scheduler uses the new WorkOrders module #1161

stuartc opened this issue Sep 28, 2023 · 2 comments
Labels
bug Newly identified bug regression Previously working feature no longer working as expected

Comments

@stuartc
Copy link
Member

stuartc commented Sep 28, 2023

Currently the Cron scheduler uses the Pipeline module, this needs to be changed to use the WorkOrders module and conform with the RTM refactor.

To see the current errors on release-0.10.0 enable a workflow with a cron trigger set to * * * * * and wait a minute.

You'll see this error:

[error] Oban exception:
%MatchError{term: {:ok, %{reason: %Lightning.InvocationReason{__meta__: #Ecto.Schema.Metadata<:loaded, "invocation_reasons">, id: "86201ee7-3ae7-4ca3-a9ff-21e4665e62e6", type: :cron, run_id: nil, run: #Ecto.Association.NotLoaded<association :run is not loaded>, user_id: nil, user: #Ecto.Association.NotLoaded<association :user is not loaded>, trigger_id: "cae544ab-03dc-4ccc-a09c-fb4edb255d7a", trigger: #Ecto.Association.NotLoaded<association :trigger is not loaded>, dataclip_id: "9c547b14-43b1-4c7d-bc68-a67200dce149", dataclip: #Ecto.Association.NotLoaded<association :dataclip is not loaded>, inserted_at: ~U[2023-11-06 13:00:00Z], updated_at: ~U[2023-11-06 13:00:00Z]}, job: %Lightning.Workflows.Job{__meta__: #Ecto.Schema.Metadata<:loaded, "jobs">, id: "f45f2f8c-beb4-49e0-9e7f-7acce956598a", body: "fn(state => state);\n", enabled: true, name: "Transform data to FHIR standard", adaptor: "@openfn/language-http@latest", project_credential_id: nil, project_credential: #Ecto.Association.NotLoaded<association :project_credential is not loaded>, credential: #Ecto.Association.NotLoaded<association :credential is not loaded>, workflow_id: "dfdf3309-b376-427c-b233-a58ef349dde9", workflow: %Lightning.Workflows.Workflow{__meta__: #Ecto.Schema.Metadata<:loaded, "workflows">, id: "dfdf3309-b376-427c-b233-a58ef349dde9", name: "OpenHIE Workflow", edges: #Ecto.Association.NotLoaded<association :edges is not loaded>, jobs: #Ecto.Association.NotLoaded<association :jobs is not loaded>, triggers: #Ecto.Association.NotLoaded<association :triggers is not loaded>, work_orders: #Ecto.Association.NotLoaded<association :work_orders is not loaded>, attempts: #Ecto.Association.NotLoaded<association :attempts is not loaded>, project_id: "4adf2644-ed4e-4f97-a24c-ab35b3cb1efa", project: #Ecto.Association.NotLoaded<association :project is not loaded>, deleted_at: nil, delete: nil, inserted_at: ~N[2023-11-06 11:35:09], updated_at: ~N[2023-11-06 12:59:34]}, project: #Ecto.Association.NotLoaded<association :project is not loaded>, delete: nil, inserted_at: ~N[2023-11-06 11:35:09], updated_at: ~N[2023-11-06 11:35:09]}, work_order: %Lightning.WorkOrder{__meta__: #Ecto.Schema.Metadata<:loaded, "work_orders">, id: "644d2838-327e-4244-b1aa-0fad0ff33f01", state: :pending, last_activity: ~U[2023-11-06 13:00:00.440768Z], workflow_id: "dfdf3309-b376-427c-b233-a58ef349dde9", workflow: %Lightning.Workflows.Workflow{__meta__: #Ecto.Schema.Metadata<:loaded, "workflows">, id: "dfdf3309-b376-427c-b233-a58ef349dde9", name: "OpenHIE Workflow", edges: #Ecto.Association.NotLoaded<association :edges is not loaded>, jobs: #Ecto.Association.NotLoaded<association :jobs is not loaded>, triggers: #Ecto.Association.NotLoaded<association :triggers is not loaded>, work_orders: #Ecto.Association.NotLoaded<association :work_orders is not loaded>, attempts: #Ecto.Association.NotLoaded<association :attempts is not loaded>, project_id: "4adf2644-ed4e-4f97-a24c-ab35b3cb1efa", project: #Ecto.Association.NotLoaded<association :project is not loaded>, deleted_at: nil, delete: nil, inserted_at: ~N[2023-11-06 11:35:09], updated_at: ~N[2023-11-06 12:59:34]}, trigger_id: nil, trigger: #Ecto.Association.NotLoaded<association :trigger is not loaded>, dataclip_id: nil, dataclip: #Ecto.Association.NotLoaded<association :dataclip is not loaded>, reason_id: "86201ee7-3ae7-4ca3-a9ff-21e4665e62e6", reason: %Lightning.InvocationReason{__meta__: #Ecto.Schema.Metadata<:loaded, "invocation_reasons">, id: "86201ee7-3ae7-4ca3-a9ff-21e4665e62e6", type: :cron, run_id: nil, run: #Ecto.Association.NotLoaded<association :run is not loaded>, user_id: nil, user: #Ecto.Association.NotLoaded<association :user is not loaded>, trigger_id: "cae544ab-03dc-4ccc-a09c-fb4edb255d7a", trigger: #Ecto.Association.NotLoaded<association :trigger is not loaded>, dataclip_id: "9c547b14-43b1-4c7d-bc68-a67200dce149", dataclip: #Ecto.Association.NotLoaded<association :dataclip is not loaded>, inserted_at: ~U[2023-11-06 13:00:00Z], updated_at: ~U[2023-11-06 13:00:00Z]}, attempts: #Ecto.Association.NotLoaded<association :attempts is not loaded>, jobs: #Ecto.Association.NotLoaded<association :jobs is not loaded>, inserted_at: ~U[2023-11-06 13:00:00.440776Z], updated_at: ~U[2023-11-06 13:00:00.440776Z]}, dataclip: %Lightning.Invocation.Dataclip{__meta__: #Ecto.Schema.Metadata<:loaded, "dataclips">, id: "9c547b14-43b1-4c7d-bc68-a67200dce149", body: %{}, type: :global, project_id: "4adf2644-ed4e-4f97-a24c-ab35b3cb1efa", project: #Ecto.Association.NotLoaded<association :project is not loaded>, source_run: #Ecto.Association.NotLoaded<association :source_run is not loaded>, inserted_at: ~U[2023-11-06 13:00:00.435516Z], updated_at: ~U[2023-11-06 13:00:00.435516Z]}}}}
    (lightning 0.10.0-pre) lib/lightning/workflows/scheduler.ex:39: anonymous fn/1 in Lightning.Workflows.Scheduler.enqueue_cronjobs/1
    (elixir 1.15.4) lib/enum.ex:984: Enum."-each/2-lists^foreach/1-0-"/2
    (lightning 0.10.0-pre) lib/lightning/workflows/scheduler.ex:38: Lightning.Workflows.Scheduler.enqueue_cronjobs/1
    (oban 2.16.2) lib/oban/queue/executor.ex:129: Oban.Queue.Executor.perform/1
    (oban 2.16.2) lib/oban/queue/executor.ex:74: Oban.Queue.Executor.call/1
    (elixir 1.15.4) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
    (elixir 1.15.4) lib/task/supervised.ex:36: Task.Supervised.reply/4

meta:
  %{
  args: %{},
  id: 1093,
  reason: %MatchError{
    term: {:ok,
     %{
       reason: %Lightning.InvocationReason{
         __meta__: #Ecto.Schema.Metadata<:loaded, "invocation_reasons">,
         id: "86201ee7-3ae7-4ca3-a9ff-21e4665e62e6",
         type: :cron,
         run_id: nil,
         run: #Ecto.Association.NotLoaded<association :run is not loaded>,
         user_id: nil,
         user: #Ecto.Association.NotLoaded<association :user is not loaded>,
         trigger_id: "cae544ab-03dc-4ccc-a09c-fb4edb255d7a",
         trigger: #Ecto.Association.NotLoaded<association :trigger is not loaded>,
         dataclip_id: "9c547b14-43b1-4c7d-bc68-a67200dce149",
         dataclip: #Ecto.Association.NotLoaded<association :dataclip is not loaded>,
         inserted_at: ~U[2023-11-06 13:00:00Z],
         updated_at: ~U[2023-11-06 13:00:00Z]
       },
       job: %Lightning.Workflows.Job{
         __meta__: #Ecto.Schema.Metadata<:loaded, "jobs">,
         id: "f45f2f8c-beb4-49e0-9e7f-7acce956598a",
         body: "fn(state => state);\n",
         enabled: true,
         name: "Transform data to FHIR standard",
         adaptor: "@openfn/language-http@latest",
         project_credential_id: nil,
         project_credential: #Ecto.Association.NotLoaded<association :project_credential is not loaded>,
         credential: #Ecto.Association.NotLoaded<association :credential is not loaded>,
         workflow_id: "dfdf3309-b376-427c-b233-a58ef349dde9",
         workflow: %Lightning.Workflows.Workflow{
           __meta__: #Ecto.Schema.Metadata<:loaded, "workflows">,
           id: "dfdf3309-b376-427c-b233-a58ef349dde9",
           name: "OpenHIE Workflow",
           edges: #Ecto.Association.NotLoaded<association :edges is not loaded>,
           jobs: #Ecto.Association.NotLoaded<association :jobs is not loaded>,
           triggers: #Ecto.Association.NotLoaded<association :triggers is not loaded>,
           work_orders: #Ecto.Association.NotLoaded<association :work_orders is not loaded>,
           attempts: #Ecto.Association.NotLoaded<association :attempts is not loaded>,
           project_id: "4adf2644-ed4e-4f97-a24c-ab35b3cb1efa",
           project: #Ecto.Association.NotLoaded<association :project is not loaded>,
           deleted_at: nil,
           delete: nil,
           inserted_at: ~N[2023-11-06 11:35:09],
           updated_at: ~N[2023-11-06 12:59:34]
         },
         project: #Ecto.Association.NotLoaded<association :project is not loaded>,
         delete: nil,
         inserted_at: ~N[2023-11-06 11:35:09],
         updated_at: ~N[2023-11-06 11:35:09]
       },
       work_order: %Lightning.WorkOrder{
         __meta__: #Ecto.Schema.Metadata<:loaded, "work_o (truncated)
@taylordowns2000
Copy link
Member

@elias-ba and @midigofrank , Nick and I just confirmed that the cron scheduler is not working properly (as described in the issue above.) Have you come across this yet in your tests/regressions work on #1213 ?

@taylordowns2000 taylordowns2000 changed the title Ensure cron scheduler uses the new WorkOrders module Make cron scheduler use the new WorkOrders module Nov 6, 2023
@taylordowns2000 taylordowns2000 changed the title Make cron scheduler use the new WorkOrders module Ensure cron scheduler uses the new WorkOrders module Nov 6, 2023
@NickOpenFn NickOpenFn added bug Newly identified bug regression Previously working feature no longer working as expected labels Nov 6, 2023
@midigofrank
Copy link
Collaborator

@taylordowns2000 yes I have come across it. The fix is in #1278, still have one more test to fix for it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Newly identified bug regression Previously working feature no longer working as expected
Projects
None yet
Development

No branches or pull requests

4 participants