Skip to content

refactor(core)!: use class decorators for explore operators and add OperatorMetadata#805

Merged
AlessandroPomponio merged 57 commits into
mainfrom
maj_operation_update
Apr 16, 2026
Merged

refactor(core)!: use class decorators for explore operators and add OperatorMetadata#805
AlessandroPomponio merged 57 commits into
mainfrom
maj_operation_update

Conversation

@michael-johnston
Copy link
Copy Markdown
Member

@michael-johnston michael-johnston commented Apr 8, 2026

Introduce ability to decorate explore operator classes directly avoiding need for a separate function and related issues due to different sources of operator metadata.

For example with this change the operator class defines the operators name; this name is what it is registered as (ado get operators); it is used to generate the operator identifier and operation identifier field values; and it is what appears in operation resource yaml . Previously these could all be set differently.

Changes

  • Extend @explore_operation to decorate classes directly. No separate function is needed (its generated automatically)
  • Migrate RandomWalk and RayTune to the class-decorator pattern; delete the now-redundant functions for both
  • Centralise operator metadata in a new OperatorMetadata model (moved to core/operation/config.py) as the single source of truth for each operator's name, version, type, configuration model, and implementation class.
  • Rename OperatorFunctionConfOperatorReference (deprecated alias kept for backward compatibility).
  • Introduce OperatorFunction a protocol based type which can be used to type and validate operator function signatures
  • Explore operations now store an OperatorReference instance in the operation resource i.e. the registered operator name rather than class name
  • orchestration functions (orchestrate_general_operation, orchestrate_explore_operation, _run_operation_harness) updated to use OperatorMetadata

Breaking

The following are not supported any more

  • explore operator classes without an implementation of the operator_metadata() class method: Explore operator classes must be updated to implement operator_metadata().
    - Instantiating an existing class that does not implement this method will cause TypeError: Can't instantiate abstract class OPERATOR_CLASS_NAME with abstract method operator_metadata
    - Calling this class method on the class (without instantiation) will cause NotImplementedError to be raised
    - Note: Via ado cli user will encounter one of the following two errors first if such a class exists.
  • decorating functions with explore_operation decorator: This decorator can only be used on classes.
    - Existing functions decorated with @explore_operation will cause TypeError: explore_operation takes 1 positional argument but N were given to be raised when the operator registry is accessed via the cli (e.g. ado get operators, ado create op)
  • Passing an operation yaml with OperatorModuleConf fields for the module field to ado create op: This was allowed but not documented anywhere as viable approach (existing only in some test files and one old example file). It's no longer allowed.
  • Not supplying values for version, configuration_model or configuration_model_default to operation function decorators other than explore_operation: These fields are now required.
    • Existing uses of these decorators that don't provide values for these fields will cause TypeError: $DECORATOR_NAME takes 4 positional arguments but (number less than 4) were given to be raised when the operator registry is accessed via the cli (e.g. ado get operators, ado create op)

Deprecation

  • OperatorFunctionConf: still importable as a deprecated alias for OperatorReference.
  • operatorIdentifier, operationType, defaultOperationParameters, validateOperationParameters of DiscoveryOperationBase - replaced by operator_metadata

Explore operator function decorator now allows registering the class that embodies the operator.

This means orchestrate_explore_operation can be passed an OperatorFunctionConf instance and retrieve the class rather than needed an OperatorModuleConf instance.
Need to use a protocol due to using kwargs
Also compress the decoration/registration to single functions
remove ray decorator if present. Will be re-added if necessary.
OperatorFunctionConf -> OperatorReference
Operator -> OperatorMetadata and moved to config.py
explore decorator -> now can be used to decorate an explore operator class
Provide fallback implementation for old explore classes.
Use default operation_metadata in the decorator
@michael-johnston michael-johnston marked this pull request as ready for review April 8, 2026 22:41
warn if two operators are loaded with same name.
@michael-johnston michael-johnston dismissed AlessandroPomponio’s stale review April 14, 2026 12:15

New updates addressing prior comments

Copy link
Copy Markdown
Member

@AlessandroPomponio AlessandroPomponio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still need to look at tests and documentation, but I noticed these things

Comment thread orchestrator/core/operation/config.py Outdated
Comment thread orchestrator/core/operation/config.py Outdated
Comment thread orchestrator/modules/operators/collections.py
@michael-johnston
Copy link
Copy Markdown
Member Author

@AlessandroPomponio last comments addressed

@AlessandroPomponio AlessandroPomponio changed the title refactor: explore operators refactor(core)!: use class decorators for explore operators and add OperatorMetadata Apr 16, 2026
Copy link
Copy Markdown
Member

@AlessandroPomponio AlessandroPomponio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks

@AlessandroPomponio AlessandroPomponio added this pull request to the merge queue Apr 16, 2026
Merged via the queue into main with commit 2494c17 Apr 16, 2026
18 of 19 checks passed
@AlessandroPomponio AlessandroPomponio deleted the maj_operation_update branch April 16, 2026 09:41
mgazz pushed a commit to mgazz/ado that referenced this pull request May 7, 2026
…perators (IBM#886)

* fix(orchestration): recursive calling of general operators

run_general_operation_core_closure was calling the decorator function not the decorated (wrapped) function. As a result this caused a recursion run_general_operation_core_closure -> orchestrate_general_operation -> run_general_operation_core_closure -> orchestrate_general_operation etc.

This was because the function passed to run_general_operation_core_closure was the wrapped function before IBM#805. This was changed to the wrapper function in IBM#805 but the actual call was not updated to account for this.

* refactor(trim): Update to access random walk for collection

Function is not present after IBM#805

* test(operators): Add a end-to-end test for general orchestration

* test(operators): add missing test file

* test(operators): add additional tests

Installing trim for orchestration tests adds two more custom experiments

* fix(tests): Missed changing assertion
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants