Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
28583a9
feat: add demo notebook for tracker features
eywalker May 3, 2025
b2ffdd3
feat: improve hashing to provide unique hash based on content
eywalker May 3, 2025
12253c4
doc: update tutorial about tracker
eywalker May 3, 2025
d445e14
Merge branch 'main' of https://github.com/walkerlab/orcabridge
eywalker May 3, 2025
632e0da
fix: improve repr
eywalker May 3, 2025
52850c8
doc: update tutorial on tracker
eywalker May 3, 2025
541e7b4
feat: improved repr for operations
eywalker May 3, 2025
0ab2b18
doc: update tracker tutorial
eywalker May 3, 2025
f4d261a
feat: add robust hashing of function based on code content
eywalker May 3, 2025
f6e871e
doc: update tutorial based on new hashing
eywalker May 3, 2025
6aa2524
feat: integrate graph drawing feature into tracker
eywalker May 3, 2025
8204aa9
feat: add pod_function decorator for simple pod function creation
eywalker May 3, 2025
0bdc325
feat: add rshift operator for simple mapping
eywalker May 3, 2025
cfae659
refactor: data storage for pod into its own class
eywalker May 3, 2025
7bcc4b6
chore: add example json files
eywalker May 3, 2025
ba0137f
refactor: change from preview to head
eywalker May 4, 2025
ff933b0
fix: repr string for MapPackets
eywalker May 4, 2025
8a5c508
fix: data store import
eywalker May 4, 2025
216b1b7
doc: update tutorial notebooks with convenience features
eywalker May 4, 2025
2e0933b
feat: add robust hashing logic for tracking and memoization
eywalker May 5, 2025
438bc21
refactor: move hashing and add simpler labeling logic
eywalker May 5, 2025
3a443cd
fix: wrong argument name used
eywalker May 5, 2025
a8f13c9
fix: superclass to Mapper
eywalker May 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions examples/dataset2/info_day1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "Day 1 experiment",
"description": "This is the first day of the experiment.",
"experimenter": "John Doe",
"date": "2023-10-01"
}
6 changes: 6 additions & 0 deletions examples/dataset2/info_day2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "Day 2 experiment",
"description": "This is the second day of the experiment.",
"experimenter": "John Doe",
"date": "2023-10-02"
}
6 changes: 6 additions & 0 deletions examples/dataset2/info_day3.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "Day 3 experiment",
"description": "This is the third day of the experiment.",
"experimenter": "Mary Smith",
"date": "2023-10-05"
}
6 changes: 6 additions & 0 deletions examples/dataset2/info_day4.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "Day 3 experiment",
"description": "This is the fourth day of the experiment.",
"experimenter": "Mary Smith",
"date": "2023-10-06"
}
6 changes: 6 additions & 0 deletions examples/dataset2/info_day5.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "Day 5 experiment",
"description": "This is the fifth day of the experiment.",
"experimenter": "Paul Brown",
"date": "2023-10-09"
}
5 changes: 0 additions & 5 deletions notebooks/01_orcabridge_core_concepts copy.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,6 @@
"* Different pipeline DAG defintion -- In `orcapod` the directed acyclic graph (DAG) for the `pipeline` should be defined using YAML file (or less frequently using API on `pipeline` struct in the Rust library). In `orcabridge` you will find that a `pipeline` DAG is defined dynamically through a series of application of `operation`. This is very much akin to how some DAG-based neural network library like TensorFlow defines a computation graph. While this works well for simple examples, it is rather difficult to track changes to the pipeline defined dynamically/programmatically using version control system. Since *how* you define the pipeline DAG is strictly speaking an orthogonal problem to the everything else that concerns the operation of the `pipeline`, no effort will be given to align the DAG definition in `orcabridge` and `orcapod`.\n",
"* Limited usage of a `stream` -- Currently `orcabrdige` only support single producer single consumer (SCSP) `stream`x, whereas in `orcapod`, `stream` should support single producer multiple consumer (SPMC) paradigm. While the same stream can be used in multiple downstream operations, each iteration of the stream actually results in recomputations of the entire chain of pipeline leading up to that stream. This inefficiency can be ameliorated by `CacheStream` operation after particularly computationally expensive segment of the pipeline. Using storage-backed `FunctionPod` will also help ameliorate the cost of recomputation by retrieving memoized computation result instead of recomputing."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
Expand Down
527 changes: 432 additions & 95 deletions notebooks/02_orcabridge_basic_usage.ipynb

Large diffs are not rendered by default.

Loading