feat(core): enable actuators and operators to use Rich progress indicators #248

michael-johnston · 2025-11-25T14:17:57Z

Actuators and operators can now display spinners and progress bars using the new RichConsoleQueue and supported message types
vllm_performance actuator and RandomWalk operator have been update to leverage this functionality

The PR also

refactors all rich.Console related code for operations (progress and results table) in a new module console_output.py
fixes issue with port-forward failure not being identified in vllm_performance

progress_example.mp4

Was printing first entity from samplestore because once this was relevant to the space - however this is no longer true and it could be anything.

…tors This allows actuators/explore operators that have steps that take significant time to execute to clearly notify users on their state, rather than relying on e.g. INFO logs which may be buried in other log output. Spinners and ProgressBars are supported in this initial implementation. Also unify all rich.Console related functionality for explore operations in a module (console_output.py)

random walk now outputs a progress bar as it processes entities - the bar also includes (number completed entities/total) and the elapsed time.

Actuator now outputs progress indicators using new RichConsoleQueue when waiting for an environment to be free; creating vllm deployment; waiting for it to be ready. Also pass requestid to create_environment so it can be referenced in the progress indicators

The code did not check if the port-forward process had exited, only if Popen raised an Exception. There are situations where the port-forward command could immediately fail that would not raise an exception. As it was the code would continue on as if the port-forward was up

… loglevel

DRL-NextGen · 2025-11-25T14:34:31Z

Checks Summary

Last run: 2025-11-26T16:31:38.177Z

Code Risk Analyzer vulnerability scan found 2 vulnerabilities:

Severity	Identifier	Package	Details	Fix
🔷Medium	CVE-2025-50181	urllib3	urllib3 redirects are not disabled when retries are disabled on PoolManager instantiation GHSA-pq67-6m6q-mj2v urllib3:2.3.0->kubernetes:34.1.0	2.5.0
🔷Medium	CVE-2025-50182	urllib3	urllib3 does not control redirects in browsers and Node.js GHSA-48p4-8xcf-vxj5 urllib3:2.3.0->kubernetes:34.1.0	2.5.0

michael-johnston · 2025-11-25T15:28:23Z

@christian-pinto can you check the changes to vllm_performance and the progress elements?

christian-pinto

Let me run it now

plugins/actuators/vllm_performance/ado_actuators/vllm_performance/experiment_executor.py

plugins/actuators/vllm_performance/yamls/vllm_deployment_space.yaml

website/docs/examples/vllm-performance-full.md

christian-pinto · 2025-11-25T16:15:47Z

Let me run it now

Looks good. THe only thing that is odd is that when a request starts running it disappears from the list. Is this intentional?

Co-authored-by: Christian Pinto <christian.pinto@ibm.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

michael-johnston · 2025-11-25T16:42:45Z

Looks good. THe only thing that is odd is that when a request starts running it disappears from the list. Is this intentional?

Yes because I thought the vllm bench serve output is there so wasn't necessary - can easily add an indicator like

(request id) running vllm bench serve ...

for when the test is running?

Edit: I added it

So an progress entry for a request is always present while running test-deployment-v1

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

christian-pinto · 2025-11-26T09:09:42Z

This is what I see now

experiment.mp4

Three experiments submitted (BS==3) but only one is shown

christian-pinto

I see the problem now. All the prints are using the definition as id. When entities require the same deployment they will all have use the same identifier for the prints.

plugins/actuators/vllm_performance/ado_actuators/vllm_performance/experiment_executor.py

orchestrator/modules/operators/_explore_orchestration.py

orchestrator/modules/operators/console_output.py

plugins/actuators/vllm_performance/ado_actuators/vllm_performance/experiment_executor.py

orchestrator/modules/operators/console_output.py

Filtering on ConstitutiveProperty type when that type is not the type of the field since intro of ConstitutivePropertyDescriptor. Use consitutive_property_values to avoid having to filter.

…rs_console

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

…ors_console

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

orchestrator/cli/resources/discovery_space/show_entities.py

…nce/experiment_executor.py Co-authored-by: Christian Pinto <christian.pinto@ibm.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

AlessandroPomponio

LGTM thanks

christian-pinto

LGTM Thanks

All requests resolved

michael-johnston added 10 commits November 21, 2025 17:44

docs(vllm_perf): example space could not be created.

7be8847

chore(logs): remove stray print

7cb6fb5

chore(logs): remove unnecessary prints

9521b87

Was printing first entity from samplestore because once this was relevant to the space - however this is no longer true and it could be anything.

feat(core): progress indicator for random walk

d54e2b8

random walk now outputs a progress bar as it processes entities - the bar also includes (number completed entities/total) and the elapsed time.

chore(vllm_performance): remove logs

88fa052

feat(vllm_performance): method to return environment usage

d1bb5e0

chore(vllm_performance): Harmonise vllm bench serve loglevel with ado…

f475ff3

… loglevel

michael-johnston requested review from AlessandroPomponio and christian-pinto November 25, 2025 14:18

michael-johnston linked an issue Nov 25, 2025 that may be closed by this pull request

feat: enable operators/actuators to write rich console live elements e.g. spinners #244

Closed

christian-pinto requested changes Nov 25, 2025

View reviewed changes

Apply suggestions from code review

b9a2b26

Co-authored-by: Christian Pinto <christian.pinto@ibm.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

michael-johnston and others added 3 commits November 25, 2025 17:58

feat(vllm_performance): Add indicator for benchmark

2da7f3f

So an progress entry for a request is always present while running test-deployment-v1

Merge branch 'main' into maj_actors_console

b40129d

Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

chore(vllm_performance): fix typing

1d3f970

michael-johnston enabled auto-merge November 26, 2025 07:20

christian-pinto previously requested changes Nov 26, 2025

View reviewed changes

AlessandroPomponio requested changes Nov 26, 2025

View reviewed changes

AlessandroPomponio changed the title ~~feat: enable actuators and operators to use Rich progress indicators~~ feat(core): enable actuators and operators to use Rich progress indicators Nov 26, 2025

chore(core): updates from review

dbf315c

michael-johnston requested a review from AlessandroPomponio November 26, 2025 12:30

refactor: simplify render_progress_indicators

8292304

AlessandroPomponio reviewed Nov 26, 2025

View reviewed changes

michael-johnston and others added 6 commits November 26, 2025 13:59

fix(core): show entities missing/unmeasured

cbf050c

Filtering on ConstitutiveProperty type when that type is not the type of the field since intro of ConstitutivePropertyDescriptor. Use consitutive_property_values to avoid having to filter.

Merge branch 'maj_fix_show_entities_missing_unmeasured' into maj_acto…

3f91907

…rs_console

Apply suggestions from code review

3639587

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

docs(website): Add docs on signalling progress from actuators

bd84556

Merge remote-tracking branch 'origin/maj_actors_console' into maj_act…

1193b0e

…ors_console

Update orchestrator/modules/operators/console_output.py

9bc0817

Co-authored-by: Alessandro Pomponio <10339005+AlessandroPomponio@users.noreply.github.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

michael-johnston requested a review from AlessandroPomponio November 26, 2025 15:45

AlessandroPomponio reviewed Nov 26, 2025

View reviewed changes

orchestrator/cli/resources/discovery_space/show_entities.py Outdated Show resolved Hide resolved

michael-johnston and others added 2 commits November 26, 2025 16:15

Merge remote-tracking branch 'origin/main' into maj_actors_console

2b07c90

Update plugins/actuators/vllm_performance/ado_actuators/vllm_performa…

bf9616c

…nce/experiment_executor.py Co-authored-by: Christian Pinto <christian.pinto@ibm.com> Signed-off-by: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>

michael-johnston requested a review from AlessandroPomponio November 26, 2025 16:17

AlessandroPomponio approved these changes Nov 26, 2025

View reviewed changes

christian-pinto approved these changes Nov 26, 2025

View reviewed changes

michael-johnston added this pull request to the merge queue Nov 26, 2025

Merged via the queue into main with commit d930308 Nov 26, 2025
18 checks passed

michael-johnston deleted the maj_actors_console branch November 26, 2025 16:53

feat(core): enable actuators and operators to use Rich progress indicators #248

feat(core): enable actuators and operators to use Rich progress indicators #248

Uh oh!

Conversation

michael-johnston commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DRL-NextGen commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michael-johnston commented Nov 25, 2025

Uh oh!

christian-pinto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christian-pinto commented Nov 25, 2025

Uh oh!

michael-johnston commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christian-pinto commented Nov 26, 2025

Uh oh!

christian-pinto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AlessandroPomponio left a comment

Choose a reason for hiding this comment

Uh oh!

christian-pinto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

michael-johnston commented Nov 25, 2025 •

edited

Loading

DRL-NextGen commented Nov 25, 2025 •

edited

Loading

michael-johnston commented Nov 25, 2025 •

edited

Loading