Skip to content

Engine: BuildFromConfig ignores yaml-level dependsOn: for external-plugin modules (init-order race) #663

@intel352

Description

@intel352

Problem

StdEngine.BuildFromConfig (engine.go:381) walks cfg.Modules in slice order (engine.go:489) and registers each module with app.RegisterModule(mod) (engine.go:517). The subsequent app.Init() (engine.go:520) then walks modules in registration order for the external-plugin path — there is no dependency-aware topological sort.

For external plugins that declare dependsOn: keys in their YAML module config, the engine validates the keys but does not honor them at init time. This means a child module's Init() can fire before its parent's Init(), breaking any plugin that uses Init() to register runtime state (broker registries, factory tables, etc.) that downstream modules look up.

Concrete impact: workflow-plugin-eventbus + BMW

BMW (buymywishlist) just shipped PR #279 switching its 6 consumers from NATS to pgchannel:

  • eventbus.broker module → registers a runtime broker via Init() (calls RegisterBrokerInstance)
  • eventbus.stream module → calls RegisterStream in Init()
  • 6 × eventbus.consumer modules → call RegisterConsumer in Init(), look up the broker via LookupRuntime(brokerRef)

With canonical names (bmw-eventbus, bmw-stream, bmw-consumer-*), alphabetical iteration fires consumers before the broker, and RegisterConsumer fails with broker not registered within 10s.

Current workaround in BMW (GoCodeAlone/buymywishlist#279, commit 765c1c6): rename broker → aaa-bmw-eventbus, stream → aab-bmw-stream so they sort lexicographically before the consumers. dependsOn: keys are kept on the consumer modules as documentation, but they are dead weight at engine init time.

This is a brittle workaround — adding any future module whose name sorts before aaa- would re-break the deploy.

Reproduction (minimal)

modules:
  - name: child
    type: eventbus.consumer  # any module whose Init() needs a parent
    config: {stream_name: x, consumer_name: y, broker_ref: parent}
    dependsOn: [parent]      # ignored
  - name: parent
    type: eventbus.broker
    config: {provider: pgchannel, broker_target: in_process, dsn: ...}

BuildFromConfig registers child then parent; app.Init() calls child.Init() first; child looks up parent in the broker registry → fails.

What we'd like

Either:

(a) Engine-level: topological sort cfg.Modules by dependsOn: before app.RegisterModule so registration order respects declared deps. (Probably cheapest; the YAML already carries the data.)

(b) Modular-level: have app.Init() honor a DependencyAware-like interface that external-plugin module factories can implement (return Dependencies() []string) and modular sorts at Init time.

(a) seems strictly better for external plugins since the dependency is a config-level property of how the operator wired the modules, not a property of the module struct.

Tag for the fix

When the engine-side fix lands, BMW will revert the aaa-/aab- prefixes in app.yaml.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions