Skip to content

Workflow event capability#19

Merged
ferglor merged 28 commits into
mainfrom
spike/beholder-capability
Nov 12, 2024
Merged

Workflow event capability#19
ferglor merged 28 commits into
mainfrom
spike/beholder-capability

Conversation

@ferglor

@ferglor ferglor commented Sep 24, 2024

Copy link
Copy Markdown
Contributor

The idea with this capability is that we want to be able to define custom events at the workflow level, using outputs of other capabilities in the event body. Some pseudo code to highlight what we think we'd need:

workflow := NewWorkflow()

httpTrigger := NewHTTPTrigger(workflow)

beholderCapability := NewBeholderCapabilitiy(workflow, map[string]any{
    "service":"httpTrigger",
    "result": httpTrigger.Output(),
})

Currently, I'm not able to define a structure in the schema that would let me construct an arbitrary map for the event data in this way; should I instead be using a compute capability to tie the event data together and feed that into the beholder capability?

Update

After some discussion with @nolag, we're now able to use an "any map" as input to the capability. This capability has been tested based on the steps outlined in this doc: https://smartcontract-it.atlassian.net/wiki/spaces/CAP/pages/1048478592/Testing+the+beholder+capability

Example workflow definition:

name: 'workfloe2'
owner: '0x00000000000000000000000000000000000000aa'
triggers:
 - id: 'cron-trigger@1.0.0'
   config:
     schedule: '*/10 * * * * *'

targets:
 - id: 'beholder-target@1.0.0'
   config:
     test: test
   inputs:
     payload:
       test: '$(trigger.outputs)'

Example use in integration/unit tests:

beholdertarget.TargetConfig{}.New(workflow, beholdertarget.TargetInput{
    Payload: sdk.AnyMap[beholdertarget.TargetInputsPayload](
        sdk.CapMap{
            "test": sdk.ConstantDefinition[string]("test"),
            "foo":  trigger.Body(),
        },
    ),
})

Comment thread beholder/target/target.go Outdated
Comment thread beholder/beholdercap/beholder_target-schema.json Outdated
@ferglor ferglor marked this pull request as ready for review September 26, 2024 13:11
@ferglor ferglor changed the title Spike: Beholder capability Beholder capability Sep 26, 2024
@ferglor ferglor requested a review from justinkaseman October 7, 2024 23:01
@ferglor ferglor force-pushed the spike/beholder-capability branch from 23a40e2 to c8c75a6 Compare October 8, 2024 01:42
@ferglor ferglor changed the base branch from main to fix/build-path October 8, 2024 01:43
Comment thread beholder/project.json Outdated
Comment thread beholder/target/target.go Outdated
Base automatically changed from fix/build-path to main October 8, 2024 15:06
Comment thread beholder/main.go
@ferglor ferglor force-pushed the spike/beholder-capability branch from 806a161 to 634a4ff Compare October 15, 2024 17:00
@ferglor ferglor requested a review from a team as a code owner October 15, 2024 17:00
Comment thread beholder/README.md Outdated
Comment thread beholder/target/target.go
Comment thread beholder/target/target.go
c.lggr.Errorw("missing payload", "workflowID", rawRequest.Metadata.WorkflowID, "executionID", rawRequest.Metadata.WorkflowExecutionID, "workflowName", rawRequest.Metadata.WorkflowName, "workflowOwner", rawRequest.Metadata.WorkflowOwner)
return capabilities.CapabilityResponse{}, errors.New("missing payload")
}
payloadMap, ok := payload.(*values.Map)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could simplify this with:

myStruct := &struct{Payload: *values.Map}{}
err := request.Inputs.UnwrapTo(myStruct)

Comment thread beholder/target/target.go
Comment thread beholder/target/target.go Outdated
Comment thread beholder/beholdercap/beholder_target-schema.json
@ferglor ferglor requested a review from nolag October 16, 2024 20:27
Comment thread beholder/target/target.go
}

if err := c.beholderClient.Emitter.Emit(ctx, bytes,
"beholder_data_schema", "/custom-message/versions/1", // required

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add beholder_domain, beholder_entity attributes
For more context https://chainlink-core.slack.com/archives/C06QQCU1SGH/p1729618194371929

@ferglor ferglor changed the title Beholder capability Workflow event capability Nov 5, 2024
"beholder_data_schema", "/custom-message/versions/1", // required
"beholder_data_type", "custom_message",
"beholder_domain", "keystone",
"beholder_entity", "values",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pkcll Are these values correct for domain and entity? From the slack thread, it seems that the entity name is derived from the name of the .proto file that defines the type; the type used here (Map) is defined in this file: https://github.com/smartcontractkit/chainlink-common/blob/main/pkg/values/pb/values.proto#L30

@ferglor ferglor force-pushed the spike/beholder-capability branch from 07c6e0e to e3f8869 Compare November 5, 2024 19:09
Comment thread workflowevent/main.go Outdated
Comment thread workflowevent/project.json Outdated
Comment thread workflowevent/target/target.go Outdated
Comment thread workflowevent/target/target_test.go Outdated

@DeividasK DeividasK left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, but overall LGTM. Happy to approve when Sonar is fixed (already pinged Alex).

Comment thread workflowevent/project.json Outdated
ferglor and others added 5 commits November 6, 2024 22:18
Co-authored-by: Deividas Karžinauskas <deividas.karzinauskas@gmail.com>
Co-authored-by: chudilka1 <yepishevsanya@gmail.com>
@ferglor ferglor requested a review from chudilka1 November 6, 2024 23:44
@ferglor ferglor requested a review from DeividasK November 7, 2024 00:27
@ferglor ferglor force-pushed the spike/beholder-capability branch from 61e926c to a55fca2 Compare November 12, 2024 02:19
@ferglor ferglor added this pull request to the merge queue Nov 12, 2024
Merged via the queue into main with commit bf33d56 Nov 12, 2024
@ferglor ferglor deleted the spike/beholder-capability branch November 12, 2024 22:14
cedric-cordenier pushed a commit that referenced this pull request Nov 19, 2024
* WIP add a beholder capability

Regenerate

Refactor

Add test coverage

Clean up project.json

goimports

Add generate task

Nx tidy

Regenerate beholder capability

Update tests

Use proto maps

Use proto marshal and unmarshal

Switch to google.golang.org/protobuf/proto

Clean up generated code

Nx generate

go mod tidy

WIP update cap

Bump common

Use the test config to disable batch, specify attributes

WIP update beholder settings

Update tests

Update test

Add readme with nx commands

Please the linter

* Use a non test config

* Update build commands

* Update the initialise function signature to allow for an OracleFactory

* go mod tidy

* Cast to values.Map and use values.ProtoMap

* goimports

* Add more logs

* Update tests, remove logs

* Please the linter

* Add example payloads to the schema

* Add attributes to the emitted message

* Rename the beholder capability to the workflowevent capability

* Fix tests

* Add beholder domain and entity, TBD

* Update tests

* Update workflowevent/project.json

Co-authored-by: Deividas Karžinauskas <deividas.karzinauskas@gmail.com>

* Use the loopserver lib

* Extract capability info

* Update workflowevent/project.json

Co-authored-by: chudilka1 <yepishevsanya@gmail.com>

* Goimports

* Use testutils

* Goimports

* go mod tidy

* Rename schema json

* Exclude main.go from coverage reports

* Make domain and entity configurable

* Use platform and values as domain and entity

---------

Co-authored-by: Sri Kidambi <1702865+kidambisrinivas@users.noreply.github.com>
Co-authored-by: Deividas Karžinauskas <deividas.karzinauskas@gmail.com>
Co-authored-by: chudilka1 <yepishevsanya@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants