|
1 | | -"""Data Model |
2 | | -
|
| 1 | +"""codeflare.pipelines.Datamodel |
3 | 2 | The core data model structures are defined here. These include the various aspects for creating a DAG, the |
4 | 3 | input and output to the DAG itself. |
5 | 4 |
|
6 | | -- :class:`~codeflare.pipelines.Datamodel.Pipeline` |
7 | | -The pipeline graph is captured in Pipeline class with the supporting constructs of Node and Edge. There are |
8 | | -different types of nodes, the key ones are EstimatorNode and an AndNode. The details of the kind of nodes |
9 | | -are captured in a separate document that also outlines the type, the firing semantics (when a node gets executed), |
10 | | -and the state of it. |
11 | | -
|
12 | | -The input and output for the pipeline are captured in the PipelineInput and PipelineOutput classes, which are |
13 | | -supported by the Xy and XYRef classes. The basic data are captured in Xy and XYRef, where XYRef is a holder for |
14 | | -pointers to X and y. The input to the pipeline defines which nodes take what Xy pointers, whereas the pipeline |
15 | | -output defines what nodes produce the outputs. |
16 | | -
|
17 | | -Finally, the data model allows for morphing of pipeline based on parameterizations, these parameterizations can |
18 | | -be for grid search or for other such similar reasons. |
| 5 | +- :class:`~codeflare.pipelines.Datamodel.Pipeline`: The class that defines the pipeline graph is captured, which is supported by Node and Edge constructs. |
| 6 | +- :class:`~codeflare.pipelines.Datamodel.Node`: The node class that has two implementations, :class:`~codeflare.pipelines.Datamodel.EstimatorNode` and :class:`~codeflare.pipelines.Datamodel.AndNode`, the details of these are captured in a separate document on nodes and their semantics defined by the type, firing semantics, and state. |
| 7 | +- :class:`~codeflare.pipelines.Datamodel.PipelineInput`: The input for pipeline, supported by Xy and XYRef holder classes. |
| 8 | +- :class:`~codeflare.pipelines.Datamodel.PipelineOutput`: The output for pipeline (after execution), supported by Xy and XYRef holder classes. |
| 9 | +- :class:`~codeflare.pipelines.Datamodel.Xy`: A basic holder class for X and y, both of which are numpy arrays. |
| 10 | +- :class:`~codeflare.pipelines.Datamodel.XYRef`: A basic holder class for pointers to X and y, pointers are reference to objects in Plasma store of Ray. |
| 11 | +- :class:`~codeflare.pipelines.Datamodel.PipelineParam`: Finally, the data model allows for morphing of pipeline based on parameterizations, these parameterizations can be for grid search or for other such similar reasons. |
19 | 12 | """ |
20 | 13 |
|
| 14 | +# Authors: Mudhakar Srivatsa <msrivats@us.ibm.com> |
| 15 | +# Raghu Ganti <rganti@us.ibm.com> |
| 16 | +# Carlos Costa <chcost@us.ibm.com> |
| 17 | +# |
| 18 | +# License: Apache v2.0 |
| 19 | + |
21 | 20 | from abc import ABC, abstractmethod |
22 | 21 | from enum import Enum |
23 | 22 |
|
|
32 | 31 |
|
33 | 32 | class Xy: |
34 | 33 | """ |
35 | | - .. _xy |
36 | | -
|
37 | 34 | Holder class for Xy, where X is array-like and y is array-like. This is the base |
38 | 35 | data structure for fully materialized X and y. |
39 | 36 |
|
|
0 commit comments