Skip to content

Meeting 2026 08 03

amirshehataornl edited this page Aug 20, 2026 · 2 revisions

Meeting - August 3, 2026

Upcoming meetings

Agenda

  • Seminar Series: Doriana Medic (University of Turin) will talk about their work on Quantum workflow tools

Attendance

Person Institution
Thomas Naughton ORNL
Michael Ferguson HPE
Francesco Medina University of Turin
Doriana Medic University of Turin
Josh Moles Snoquera
Brandon Neth HPE
Abdur Rahman Hatim IISc
Philipp Seitz Quantinuum
Eun Kyung Lee IBM
Munetaka Ohtani IBM
Bran Radovanovic ORNL
Claudio Siqueira de Carvalho IBM
Ryan Hoffman Quantinuum
Peter Groszkowski ORNL
John Children Quantinuum
Michael Sandoval ORNL
Andy Stone HPE
Dan Holme Qoro
Burns Healy Tyto Athene
Sakshi Chhabra IISc
Yogesh Simmhan IISc
Johannes Blaschke EIT
Sebastian Stern Amazon
Yoonho Park IBM
Gabriella Bettonte E4
Alain Roy IonQ
Sangram Shivajirao Deshpande North Carolina State University
Narasinga Rao Miniskar ORNL
Smriti Bajaj Dell Technologies
Elaine Wong ORNL
Alex Chernoguzov Quantinuum
Masoud Mohseni HPE
Namit Anand HPE
Simone Rizzo E4
Muhammad Osama AMD
Matthieu Moreau PASQAL
Aaron Lott HPE
Jeff Heckey AWS
Jonathan Skone NERSC/LBL
Cameron Brunner Siemens DISW

Executive Summary

  • Doriana Medic from the University of Turin presented work on workflow orchestration for heterogeneous HPC, cloud, AI, and quantum computing environments.
  • The talk motivated workflows as a way to manage modern applications that are no longer monolithic, but are pipelines of data movement, simulation, machine learning, optimization, visualization, and post-processing steps.
  • Doriana described hybrid workflows as a separation between the workflow model and the deployment environment, allowing tasks to be mapped to different locations without rewriting the application logic.
  • The University of Turin group developed Streamflow, a CWL-compliant, container-native workflow management system with modular connectors for different execution environments.
  • QSplit was presented as a workflow-oriented hybrid quantum-classical optimization framework that decomposes QUBO problems, dispatches subproblems to different backends, and aggregates partial results.
  • The group discussed limitations of naive QUBO and circuit-cutting decompositions, especially the need to account for correlations and problem structure rather than only load balancing.
  • Doriana also described decentralized orchestration and SWIR, an intermediate representation intended to compile workflow descriptions into distributed execution bundles.
  • The Q&A focused on circuit cutting, decomposition strategy, dynamic scheduling, fault-tolerance primitives, backend-specific information, and how quantum calibration events might trigger workflow loops or recompilation.

Notes

  • NOTICE: Notes generated using AI; please verify accuracy.

1. Opening

  • Thomas Naughton opened the seminar and introduced Doriana Medic from the University of Turin.
  • Doriana introduced herself as an assistant professor in the Computer Science Department at the University of Turin.
  • She said she is part of the Alpha Parallel Computing Group, a heterogeneous team covering HPC, parallel programming, RISC-V and next-generation architectures, system software, data-intensive computing, cloud, federated learning, distributed AI, and quantum computing.
  • The talk focused on distributed workflows and the intersection between workflow systems and quantum computing.

2. Motivation: Heterogeneous Computing Ecosystems

  • Doriana described a major shift in computing demand:
    • HPC is still needed for simulation and engineering.
    • AI, foundational models, AI for science, and data-intensive workflows are creating new demand.
    • Quantum processors are increasingly available through cloud providers, national infrastructures, and co-location with HPC systems.
  • She noted that Europe is investing heavily through EuroHPC, AI factories, quantum initiatives, and national centers such as Italy's National Centre for HPC, Big Data, and Quantum Computing.
  • The resulting computing landscape is increasingly heterogeneous, combining CPUs, GPUs, cloud resources, AI accelerators, quantum processors, edge resources, and specialized hardware.
  • Doriana emphasized that access to these resources does not automatically make applications easier or faster; orchestration becomes the core challenge.
  • Modern scientific and industrial applications are often complex pipelines, not single programs.
  • Different stages in a pipeline may need different execution environments:
    • Simulation on HPC.
    • Training on GPUs.
    • Data processing on cloud or edge systems.
    • Optimization on quantum resources.
  • The challenge is transparent, efficient, reproducible, and portable orchestration across that ecosystem.

3. Workflow Model and Hybrid Workflows

  • Doriana described workflows as collections of interconnected steps with data inputs, outputs, and control dependencies.
  • Scientific workflows are often modeled as directed acyclic graphs, though some modern use cases such as federated learning may include cycles or loops.
  • Workflow components include:
    • Tasks or steps.
    • Ports for input/output locations.
    • Data.
    • Dependencies between tasks.
  • A workflow model describes what the application does, but not where it runs.
  • Hybrid workflows add deployment information:
    • Locations or computing environments.
    • Topology between locations.
    • Mapping of workflow steps to resources.
  • This separation allows a task to be moved from one location to another by changing the mapping rather than rewriting the workflow logic.

4. Centralized Orchestration and Streamflow

  • Doriana described centralized orchestration as a model where one orchestration engine manages the whole workflow.
  • Advantages include:
    • A global view of workflow state.
    • More informed scheduling decisions.
    • Easier monitoring and logging.
    • Reuse of mature workflow frameworks.
  • Disadvantages include:
    • A single point of failure.
    • Communication overhead between the orchestrator and execution sites.
    • Less autonomy for individual execution sites.
  • The University of Turin group developed Streamflow for this model.
  • Streamflow is:
    • CWL-compliant.
    • Container-native.
    • Designed for workflows across heterogeneous environments.
    • Modular, with new environments added through connectors rather than core changes.
  • Existing and developing connectors include Slurm, Apptainer, Docker, and quantum backends.
  • Doriana described the Common Workflow Language as an open standard for describing command-line tools and reproducible workflows.
  • CWL describes inputs, outputs, parameters, and dependencies, but does not itself decide deployment placement.
  • Streamflow use cases included:
    • Cross-facility federated learning across language-specific LLM training environments and cloud aggregation.
    • Machine-learning workflows spanning private environments, cloud, and HPC for cadastral image analysis.
    • Inclusion in an Italian AI factory environment.

5. QSplit

  • Doriana presented QSplit as a workflow-oriented hybrid quantum-classical optimization framework built around Streamflow.
  • QSplit targets problems expressed in QUBO form.
  • Its goal is to address current quantum-device limits and large problem instances by decomposing large optimization problems.
  • The QSplit workflow:
    • Takes a dataset of QUBO matrices.
    • Dispatches each instance into a solver pipeline.
    • Splits large QUBO instances into smaller matrices.
    • Solves sub-instances on different backends.
    • Aggregates partial results into a global solution.
  • Backends can include:
    • Quantum simulators on HPC.
    • HPC CPU solvers.
    • HPC GPU solvers.
    • Real QPUs, including IQM in the presented work.
  • Lightweight dispatch and collect tasks can run on cloud resources.
  • More computationally intensive splitting and aggregation tasks run on HPC.
  • Doriana said early max-cut cases with sparse matrices gave results reasonably consistent with simulated annealing, while denser or more heavily split problems showed larger deviation.
  • She emphasized that QSplit currently demonstrates orchestration of decomposition, dispatch, execution, and aggregation; it is not itself a new optimization algorithm.

6. QUBO Decomposition Discussion

  • Masoud Mohseni asked how QSplit decomposes problems into subproblems and whether it identifies quantum-prone subproblems or uses a more generic splitting routine.
  • Doriana said the current implementation recursively splits the QUBO matrix into smaller blocks until they reach the desired size.
  • Masoud characterized that as closer to load-balanced decomposition than structure-aware decomposition.
  • He cautioned that QUBO decomposition is highly non-trivial because variables may have strong graph, frustration, or correlation structure.
  • Masoud also cautioned against treating QUBO formulation as a neutral representation:
    • Embedding higher-order problems into QUBO can change the complexity of the problem.
    • Embedding can introduce correlations and alter relevant gaps, not merely add overhead.
    • It is better to stay close to the native problem formulation when possible.
  • Doriana clarified that QSplit currently uses QUBO because it is the chosen case study, but the workflow could call other scripts or decomposition strategies if provided.
  • She said future work includes adding decomposition strategies and making QSplit more parametric with respect to the strategy used.

7. Decentralized Orchestration and SWIR

  • Doriana contrasted centralized orchestration with decentralized orchestration.
  • In decentralized orchestration, workflow control and decisions are distributed across multiple components rather than managed by one central orchestrator.
  • Advantages include:
    • Less dependence on a central controller.
    • Lower communication overhead.
    • Faster local decisions.
    • Greater autonomy for execution sites.
    • Better fit for edge or highly distributed environments.
  • Challenges include:
    • Harder global optimization.
    • Harder coordination.
    • More difficult fault tolerance because no single component has the full workflow state.
  • The group developed SWIR, a scientific workflow intermediate representation, to bridge high-level workflow descriptions and low-level execution environments.
  • The compiler currently translates Pegasus workflows, with CWL support under development.
  • SWIR represents each location's trace using send/receive-style actions and can be optimized.
  • It can be compiled into a distributed executable bundle tailored to the target environment.
  • The bundle contains the information each location needs, avoiding the need for a central controller during execution.
  • The group is also working on agent-based workflow control using policy constraints and formal semantics.
  • Doriana said adding AI agents to workflow management raises verification questions: autonomous decisions must preserve the user's intent and policies across the full workflow lifecycle.

8. Future Work

  • Doriana listed several future directions:
    • Add new backends, including Quantinuum and PASQAL.
    • Add new decomposition strategies.
    • Extend QSplit to additional use cases beyond QUBO optimization.
    • Explore circuit cutting as a workflow-orchestration use case.
    • Apply the framework to multi-QPU and multi-HPC scenarios.
    • Explore quantum machine learning and quantum optimization workflows.
    • Extend SWIR and complete CWL-to-SWIR translation.
  • She described circuit cutting as analogous to the existing QSplit pattern:
    • Cut the circuit.
    • Dispatch pieces to different backends.
    • Aggregate results.

9. Q&A: Circuit Cutting and Correlation Structure

  • Namit Anand asked about the group's quantum circuit-cutting strategy.
  • Doriana said the circuit-cutting work is still early; QSplit currently targets QUBO problems, but the group is interested in expanding it to circuit-cutting pipelines.
  • Namit said HPE thinks about circuit cutting often, including using entanglement as a guiding metric, and offered to discuss further.
  • Doriana said the group is especially interested in orchestrating the full circuit-cutting pipeline, including multi-QPU and multi-HPC execution.
  • Masoud emphasized that circuit cutting should not be done with naive load balancing or gate-by-gate heuristics.
  • He said structure-aware cutting can produce large improvements compared with naive cutting:
    • Quadratic reductions in exponents for some electronic-structure cases.
    • Quadratic reductions in quantum-interconnect requirements.
    • Potentially exponential improvements for transverse-Ising-style models.
  • Masoud said cuts must account for classical and quantum correlations; cutting strongly communicating blocks can impose exponential costs.
  • He also noted that some cuts may need to be adaptive and performed in real time.

10. Q&A: Fault Tolerance, Dynamic Scheduling, and Calibration

  • Thomas Naughton asked whether the decentralized workflow system needs new primitives such as consensus, or whether those can be represented as normal workflow nodes.
  • Doriana said that for now they are represented as nodes, but she expects additional mechanisms may be needed.
  • Thomas asked how the system surfaces dynamic behavior from resources, such as slowdowns, changing availability, or connectivity changes, and how that feeds scheduling decisions.
  • Doriana said Streamflow already has a scheduler that can react to some dynamic conditions:
    • If a location is down.
    • If queue conditions make another location preferable.
    • If topology constrains data movement between locations.
  • She said the current quantum integration is less mature:
    • New quantum connectors and scheduling policies were added.
    • Current assumptions are closer to "everything is connected."
    • Backend-specific information from Quantinuum, PASQAL, and other systems needs better integration.
  • Thomas asked whether quantum calibration events could trigger revisiting earlier workflow steps such as compilation.
  • Doriana said QSplit/Streamflow does not currently have a primitive for calibration events, but such behavior could be modeled as a loop triggered by events.
  • Thomas noted that quantum-specific dynamic events could be an interesting use case for workflow systems.

11. Key Takeaways

  • Workflow orchestration is a central systems problem for quantum-HPC integration because applications increasingly span multiple heterogeneous resources.
  • Separating workflow logic from deployment mapping improves portability and makes it easier to move tasks across resources.
  • Streamflow provides a CWL-compliant, connector-based centralized orchestration model for heterogeneous workflows.
  • QSplit demonstrates a workflow-oriented approach to decomposing and dispatching hybrid quantum-classical optimization tasks.
  • QUBO splitting and circuit cutting must become structure-aware; naive load-balanced decomposition can miss important correlations and create large hidden costs.
  • Decentralized orchestration and intermediate representations such as SWIR may reduce overhead and improve autonomy, but introduce hard coordination and fault-tolerance questions.
  • Quantum-specific events such as calibration changes and backend availability need to be integrated into workflow scheduling models.

Files

Clone this wiki locally