From 4c420aca5b467e97f046641aa84065de84920517 Mon Sep 17 00:00:00 2001 From: frthjf Date: Fri, 24 May 2024 17:29:29 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20machinab?= =?UTF-8?q?le-org/machinable@299ea20d37effd4845f0cc6620ae54ce17ab6d06=20?= =?UTF-8?q?=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- 404.html | 2 +- about/approach.html | 2 +- ....js => examples_aimstack-storage_index.md.c3418677.js} | 2 +- ...> examples_aimstack-storage_index.md.c3418677.lean.js} | 0 ...33.js => examples_globus-storage_index.md.a8bc1101.js} | 2 +- ... => examples_globus-storage_index.md.a8bc1101.lean.js} | 0 ...2d6.js => examples_mpi-execution_index.md.9dd8e138.js} | 2 +- ...s => examples_mpi-execution_index.md.9dd8e138.lean.js} | 0 ...js => examples_require-execution_index.md.5e46839f.js} | 2 +- ... examples_require-execution_index.md.5e46839f.lean.js} | 0 ...3.js => examples_slurm-execution_index.md.68c750f8.js} | 2 +- ...=> examples_slurm-execution_index.md.68c750f8.lean.js} | 0 ...nent.md.eef0762f.js => guide_component.md.d7585b40.js} | 2 +- ...f0762f.lean.js => guide_component.md.d7585b40.lean.js} | 0 ...face.md.8d3c5449.js => guide_interface.md.03f9e498.js} | 4 ++-- ...3c5449.lean.js => guide_interface.md.03f9e498.lean.js} | 2 +- ...n.md.56946a6b.js => guide_introduction.md.619bab14.js} | 4 ++-- ...a6b.lean.js => guide_introduction.md.619bab14.lean.js} | 0 examples/aimstack-storage/index.html | 6 +++--- examples/dependent-schedules/index.html | 2 +- examples/globus-storage/index.html | 6 +++--- examples/index.html | 2 +- examples/mpi-execution/index.html | 6 +++--- examples/require-execution/index.html | 6 +++--- examples/slurm-execution/index.html | 6 +++--- guide/component.html | 6 +++--- guide/element.html | 2 +- guide/installation.html | 2 +- guide/interface.html | 6 +++--- guide/introduction.html | 8 ++++---- hashmap.json | 2 +- index.html | 2 +- reference/index.html | 2 +- 33 files changed, 45 insertions(+), 45 deletions(-) rename assets/{examples_aimstack-storage_index.md.6eacb297.js => examples_aimstack-storage_index.md.c3418677.js} (99%) rename assets/{examples_aimstack-storage_index.md.6eacb297.lean.js => examples_aimstack-storage_index.md.c3418677.lean.js} (100%) rename assets/{examples_globus-storage_index.md.f1237033.js => examples_globus-storage_index.md.a8bc1101.js} (99%) rename assets/{examples_globus-storage_index.md.f1237033.lean.js => examples_globus-storage_index.md.a8bc1101.lean.js} (100%) rename assets/{examples_mpi-execution_index.md.bf53f2d6.js => examples_mpi-execution_index.md.9dd8e138.js} (99%) rename assets/{examples_mpi-execution_index.md.bf53f2d6.lean.js => examples_mpi-execution_index.md.9dd8e138.lean.js} (100%) rename assets/{examples_require-execution_index.md.08f79fdd.js => examples_require-execution_index.md.5e46839f.js} (98%) rename assets/{examples_require-execution_index.md.08f79fdd.lean.js => examples_require-execution_index.md.5e46839f.lean.js} (100%) rename assets/{examples_slurm-execution_index.md.939e1383.js => examples_slurm-execution_index.md.68c750f8.js} (99%) rename assets/{examples_slurm-execution_index.md.939e1383.lean.js => examples_slurm-execution_index.md.68c750f8.lean.js} (100%) rename assets/{guide_component.md.eef0762f.js => guide_component.md.d7585b40.js} (99%) rename assets/{guide_component.md.eef0762f.lean.js => guide_component.md.d7585b40.lean.js} (100%) rename assets/{guide_interface.md.8d3c5449.js => guide_interface.md.03f9e498.js} (98%) rename assets/{guide_interface.md.8d3c5449.lean.js => guide_interface.md.03f9e498.lean.js} (97%) rename assets/{guide_introduction.md.56946a6b.js => guide_introduction.md.619bab14.js} (97%) rename assets/{guide_introduction.md.56946a6b.lean.js => guide_introduction.md.619bab14.lean.js} (100%) diff --git a/404.html b/404.html index fb0672e7..a7eee853 100644 --- a/404.html +++ b/404.html @@ -21,7 +21,7 @@
machinable

Appearance

404

PAGE NOT FOUND

But if you don't change your direction, and if you keep looking, you may end up where you are heading.
Take me home

MIT Licensed

Copyright © 2021-present

- diff --git a/about/approach.html b/about/approach.html index 75c3127f..37d8b9d2 100644 --- a/about/approach.html +++ b/about/approach.html @@ -25,7 +25,7 @@
machinable

Appearance

On this page

About machinable's approach ​

Optional reading

This background discusses the big-picture approach. For a hands-on tutorial, refer to the guide.

Coming soon

This section is currently under construction

MIT Licensed

Copyright © 2021-present

- diff --git a/assets/examples_aimstack-storage_index.md.6eacb297.js b/assets/examples_aimstack-storage_index.md.c3418677.js similarity index 99% rename from assets/examples_aimstack-storage_index.md.6eacb297.js rename to assets/examples_aimstack-storage_index.md.c3418677.js index 169bedee..f586e155 100644 --- a/assets/examples_aimstack-storage_index.md.6eacb297.js +++ b/assets/examples_aimstack-storage_index.md.c3418677.js @@ -2,7 +2,7 @@ import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C= get("aimstack", {"repo": "./path/to/aim-repo"}).__enter__() -# your code

Source ​

py
from typing import List, Optional
+# your code

Source ​

py
from typing import List, Optional
 
 import os
 
diff --git a/assets/examples_aimstack-storage_index.md.6eacb297.lean.js b/assets/examples_aimstack-storage_index.md.c3418677.lean.js
similarity index 100%
rename from assets/examples_aimstack-storage_index.md.6eacb297.lean.js
rename to assets/examples_aimstack-storage_index.md.c3418677.lean.js
diff --git a/assets/examples_globus-storage_index.md.f1237033.js b/assets/examples_globus-storage_index.md.a8bc1101.js
similarity index 99%
rename from assets/examples_globus-storage_index.md.f1237033.js
rename to assets/examples_globus-storage_index.md.a8bc1101.js
index ccc3b92e..09876b0a 100644
--- a/assets/examples_globus-storage_index.md.f1237033.js
+++ b/assets/examples_globus-storage_index.md.a8bc1101.js
@@ -13,7 +13,7 @@ import{_ as s,o as n,c as a,O as l}from"./chunks/framework.62020867.js";const i=
 print(matches)
 
 # download from globus
-storage.download(matches[0].uuid)

Source ​

py
from typing import TYPE_CHECKING, List
+storage.download(matches[0].uuid)

Source ​

py
from typing import TYPE_CHECKING, List
 
 import os
 import time
diff --git a/assets/examples_globus-storage_index.md.f1237033.lean.js b/assets/examples_globus-storage_index.md.a8bc1101.lean.js
similarity index 100%
rename from assets/examples_globus-storage_index.md.f1237033.lean.js
rename to assets/examples_globus-storage_index.md.a8bc1101.lean.js
diff --git a/assets/examples_mpi-execution_index.md.bf53f2d6.js b/assets/examples_mpi-execution_index.md.9dd8e138.js
similarity index 99%
rename from assets/examples_mpi-execution_index.md.bf53f2d6.js
rename to assets/examples_mpi-execution_index.md.9dd8e138.js
index 70514ec1..669a3e0b 100644
--- a/assets/examples_mpi-execution_index.md.bf53f2d6.js
+++ b/assets/examples_mpi-execution_index.md.9dd8e138.js
@@ -1,7 +1,7 @@
 import{_ as s,o as n,c as a,O as l}from"./chunks/framework.62020867.js";const i=JSON.parse('{"title":"MPI execution","description":"","frontmatter":{},"headers":[],"relativePath":"examples/mpi-execution/index.md","filePath":"examples/mpi-execution/index.md"}'),p={name:"examples/mpi-execution/index.md"},o=l(`
MPI execution ​
Integration to launch MPI jobs.
Usage example ​
python
from machinable import get
 
 with get("mpi", {"ranks": 8}):
-    ... # your MPI ready component
Source ​
py
from typing import Literal, Optional, Union
+    ... # your MPI ready component
Source ​
py
from typing import Literal, Optional, Union
 
 import sys
 
diff --git a/assets/examples_mpi-execution_index.md.bf53f2d6.lean.js b/assets/examples_mpi-execution_index.md.9dd8e138.lean.js
similarity index 100%
rename from assets/examples_mpi-execution_index.md.bf53f2d6.lean.js
rename to assets/examples_mpi-execution_index.md.9dd8e138.lean.js
diff --git a/assets/examples_require-execution_index.md.08f79fdd.js b/assets/examples_require-execution_index.md.5e46839f.js
similarity index 98%
rename from assets/examples_require-execution_index.md.08f79fdd.js
rename to assets/examples_require-execution_index.md.5e46839f.js
index 260f0fff..011d7612 100644
--- a/assets/examples_require-execution_index.md.08f79fdd.js
+++ b/assets/examples_require-execution_index.md.5e46839f.js
@@ -1,7 +1,7 @@
 import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=JSON.parse('{"title":"Require execution","description":"","frontmatter":{},"headers":[],"relativePath":"examples/require-execution/index.md","filePath":"examples/require-execution/index.md"}'),p={name:"examples/require-execution/index.md"},o=l(`
Require execution ​
A way to assert that components have been cached.
Usage example ​
python
from machinable import get
 
 with get("require"):
-    ... # components to check
Source ​
py
from machinable import Execution
+    ... # components to check
Source ​
py
from machinable import Execution
 
 
 class Require(Execution):
diff --git a/assets/examples_require-execution_index.md.08f79fdd.lean.js b/assets/examples_require-execution_index.md.5e46839f.lean.js
similarity index 100%
rename from assets/examples_require-execution_index.md.08f79fdd.lean.js
rename to assets/examples_require-execution_index.md.5e46839f.lean.js
diff --git a/assets/examples_slurm-execution_index.md.939e1383.js b/assets/examples_slurm-execution_index.md.68c750f8.js
similarity index 99%
rename from assets/examples_slurm-execution_index.md.939e1383.js
rename to assets/examples_slurm-execution_index.md.68c750f8.js
index 12102874..f616bfee 100644
--- a/assets/examples_slurm-execution_index.md.939e1383.js
+++ b/assets/examples_slurm-execution_index.md.68c750f8.js
@@ -1,7 +1,7 @@
 import{_ as s,o as n,c as a,O as l}from"./chunks/framework.62020867.js";const i=JSON.parse('{"title":"Slurm execution","description":"","frontmatter":{},"headers":[],"relativePath":"examples/slurm-execution/index.md","filePath":"examples/slurm-execution/index.md"}'),o={name:"examples/slurm-execution/index.md"},p=l(`
Slurm execution ​
Integration to submit to the Slurm scheduler.
Usage example ​
python
from machinable import get
 
 with get("slurm", {"ranks": 8, 'preamble': 'mpirun'}):
-    ... # your component
Source ​
py
from typing import Literal, Optional, Union
+    ... # your component
Source ​
py
from typing import Literal, Optional, Union
 
 import os
 import subprocess
diff --git a/assets/examples_slurm-execution_index.md.939e1383.lean.js b/assets/examples_slurm-execution_index.md.68c750f8.lean.js
similarity index 100%
rename from assets/examples_slurm-execution_index.md.939e1383.lean.js
rename to assets/examples_slurm-execution_index.md.68c750f8.lean.js
diff --git a/assets/guide_component.md.eef0762f.js b/assets/guide_component.md.d7585b40.js
similarity index 99%
rename from assets/guide_component.md.eef0762f.js
rename to assets/guide_component.md.d7585b40.js
index b29f4493..b4ac01fa 100644
--- a/assets/guide_component.md.eef0762f.js
+++ b/assets/guide_component.md.d7585b40.js
@@ -17,7 +17,7 @@ import{_ as p,D as t,o as c,c as r,z as n,a as s,G as o,B as l,O as e}from"./chu
 >>> mnist.launch()
 Downloading 'mnist' ...
If the execution is successful, the component is marked as finished.
python
>>> mnist.execution.is_finished()
 True
By design, component instances can only be executed once. They are automatically assigned a timestamp, random seed, as well as a nickname for easy identification.
python
>>> mnist.seed
-1632827863
Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.
Implementing custom execution ​
Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.
`,8),C=e(`
python
from multiprocessing import Pool
+1632827863
Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.
Implementing custom execution ​
Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.
`,8),C=e(`
python
from multiprocessing import Pool
 
 from machinable import Execution
 
diff --git a/assets/guide_component.md.eef0762f.lean.js b/assets/guide_component.md.d7585b40.lean.js
similarity index 100%
rename from assets/guide_component.md.eef0762f.lean.js
rename to assets/guide_component.md.d7585b40.lean.js
diff --git a/assets/guide_interface.md.8d3c5449.js b/assets/guide_interface.md.03f9e498.js
similarity index 98%
rename from assets/guide_interface.md.8d3c5449.js
rename to assets/guide_interface.md.03f9e498.js
index 644173c1..50e13bfc 100644
--- a/assets/guide_interface.md.8d3c5449.js
+++ b/assets/guide_interface.md.03f9e498.js
@@ -21,7 +21,7 @@ import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chu
 ├─ estimate_gravity.py            # contains a data analysis component
 ├─ evolution/                   
 |  └─ simulate_offspring.py       # contains a evolutionary simulation
-└─ main.py                        # main script to execute
The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:
python
from machinable import get
+└─ main.py                        # main script to execute
The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:
python
from machinable import get
 
 from estimate_gravity import EstimateGravity
 from evolution.simulate_offspring import SimulateOffspring
@@ -40,4 +40,4 @@ import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chu
 mnist = get("mnist_data")
 with open(mnist.local_directory("download_script.sh"), "w") as f:
     f.write(...)
-    os.chmod(f.name, 0o755)
Overall, interfaces make it easy to associate data with code as instantiation, storage and retrieval are managed automatically behind the scenes.
`,2);function _(b,v,E,q,T,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),u,s(" of "),h,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
+    os.chmod(f.name, 0o755)
Overall, interfaces make it easy to associate data with code as instantiation, storage and retrieval are managed automatically behind the scenes.
`,2);function _(b,v,q,E,T,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),u,s(" of "),h,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
diff --git a/assets/guide_interface.md.8d3c5449.lean.js b/assets/guide_interface.md.03f9e498.lean.js
similarity index 97%
rename from assets/guide_interface.md.8d3c5449.lean.js
rename to assets/guide_interface.md.03f9e498.lean.js
index 48e28fb1..715e6835 100644
--- a/assets/guide_interface.md.8d3c5449.lean.js
+++ b/assets/guide_interface.md.03f9e498.lean.js
@@ -1 +1 @@
-import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chunks/framework.62020867.js";const k=JSON.parse('{"title":"Interface","description":"","frontmatter":{},"headers":[],"relativePath":"guide/interface.md","filePath":"guide/interface.md"}'),i={name:"guide/interface.md"},y=a("h1",{id:"interface",tabindex:"-1"},[s("Interface "),a("a",{class:"header-anchor",href:"#interface","aria-label":'Permalink to "Interface"'},"​")],-1),D=a("p",null,[a("a",{href:"./element.html"},"Elements"),s(" by themselves are limited in that they are effectively stateless. You can construct and use them but any computed result or additional information will not be persisted.")],-1),F=e("",6),A=a("code",null,"MnistData()",-1),d=e("",3),C=a("code",null,"MnistData",-1),u=a("code",null,"batch_size",-1),h=a("code",null,"8",-1),m=e("",7),f=e("",1),g=e("",2);function _(b,v,E,q,T,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),u,s(" of "),h,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
+import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chunks/framework.62020867.js";const k=JSON.parse('{"title":"Interface","description":"","frontmatter":{},"headers":[],"relativePath":"guide/interface.md","filePath":"guide/interface.md"}'),i={name:"guide/interface.md"},y=a("h1",{id:"interface",tabindex:"-1"},[s("Interface "),a("a",{class:"header-anchor",href:"#interface","aria-label":'Permalink to "Interface"'},"​")],-1),D=a("p",null,[a("a",{href:"./element.html"},"Elements"),s(" by themselves are limited in that they are effectively stateless. You can construct and use them but any computed result or additional information will not be persisted.")],-1),F=e("",6),A=a("code",null,"MnistData()",-1),d=e("",3),C=a("code",null,"MnistData",-1),u=a("code",null,"batch_size",-1),h=a("code",null,"8",-1),m=e("",7),f=e("",1),g=e("",2);function _(b,v,q,E,T,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),u,s(" of "),h,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
diff --git a/assets/guide_introduction.md.56946a6b.js b/assets/guide_introduction.md.619bab14.js
similarity index 97%
rename from assets/guide_introduction.md.56946a6b.js
rename to assets/guide_introduction.md.619bab14.js
index c0120540..bdbb884c 100644
--- a/assets/guide_introduction.md.56946a6b.js
+++ b/assets/guide_introduction.md.619bab14.js
@@ -1,4 +1,4 @@
-import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=JSON.parse('{"title":"Introduction","description":"","frontmatter":{},"headers":[],"relativePath":"guide/introduction.md","filePath":"guide/introduction.md"}'),o={name:"guide/introduction.md"},p=l(`
Introduction ​
What is machinable? ​
machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.
The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.
  1. Write some code
python
from random import random
+import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=JSON.parse('{"title":"Introduction","description":"","frontmatter":{},"headers":[],"relativePath":"guide/introduction.md","filePath":"guide/introduction.md"}'),o={name:"guide/introduction.md"},p=l(`
Introduction ​
What is machinable? ​
machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.
The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.
  1. Write some code
python
from random import random
 
 from pydantic import BaseModel, Field
 
@@ -26,7 +26,7 @@ import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=
             print(
                 f"After {self.config.samples} samples, "
                 f"PI is approximately {self.load_file('result.json')['pi']}."
-            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
+            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
 
 # Imports component in \`montecarlo.py\` with samples=150;
 # if an component with this configuration exists, it
diff --git a/assets/guide_introduction.md.56946a6b.lean.js b/assets/guide_introduction.md.619bab14.lean.js
similarity index 100%
rename from assets/guide_introduction.md.56946a6b.lean.js
rename to assets/guide_introduction.md.619bab14.lean.js
diff --git a/examples/aimstack-storage/index.html b/examples/aimstack-storage/index.html
index 0672b20c..33f4dfdb 100644
--- a/examples/aimstack-storage/index.html
+++ b/examples/aimstack-storage/index.html
@@ -11,7 +11,7 @@
   
   
   
-  
+  
   
   
   
@@ -28,7 +28,7 @@
 
 get("aimstack", {"repo": "./path/to/aim-repo"}).__enter__()
 
-# your code
Source ​
py
from typing import List, Optional
+# your code
Source ​
py
from typing import List, Optional
 
 import os
 
@@ -84,7 +84,7 @@
 
         for k, v in interface.__model__.model_dump().items():
             run[k] = v
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/examples/dependent-schedules/index.html b/examples/dependent-schedules/index.html
index 6af728ca..f98b46ed 100644
--- a/examples/dependent-schedules/index.html
+++ b/examples/dependent-schedules/index.html
@@ -25,7 +25,7 @@
   
   
     
machinable
Appearance
On this page
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/examples/globus-storage/index.html b/examples/globus-storage/index.html
index 016874f2..d8bb422c 100644
--- a/examples/globus-storage/index.html
+++ b/examples/globus-storage/index.html
@@ -11,7 +11,7 @@
   
   
   
-  
+  
   
   
   
@@ -39,7 +39,7 @@
 print(matches)
 
 # download from globus
-storage.download(matches[0].uuid)
Source ​
py
from typing import TYPE_CHECKING, List
+storage.download(matches[0].uuid)
Source ​
py
from typing import TYPE_CHECKING, List
 
 import os
 import time
@@ -265,7 +265,7 @@
                 found.append(item["name"])
 
         return found

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/index.html b/examples/index.html index c0a9f589..8743a6f0 100644 --- a/examples/index.html +++ b/examples/index.html @@ -25,7 +25,7 @@
machinable

Appearance

On this page

How-to examples ​

This section presents code examples that demonstrate real-world usage and my be a good starting point when implementing your own projects.

Please select an example from the menu.

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/mpi-execution/index.html b/examples/mpi-execution/index.html index 5ecfaab1..38d819d5 100644 --- a/examples/mpi-execution/index.html +++ b/examples/mpi-execution/index.html @@ -11,7 +11,7 @@ - + @@ -27,7 +27,7 @@
machinable

Appearance

On this page

MPI execution ​

Integration to launch MPI jobs.

Usage example ​

python
from machinable import get
 
 with get("mpi", {"ranks": 8}):
-    ... # your MPI ready component

Source ​

py
from typing import Literal, Optional, Union
+    ... # your MPI ready component

Source ​

py
from typing import Literal, Optional, Union
 
 import sys
 
@@ -151,7 +151,7 @@
                     raise KeyboardInterrupt(
                         "Interrupting `" + " ".join(cmd) + "`"
                     ) from _ex

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/require-execution/index.html b/examples/require-execution/index.html index 5706c215..c187681e 100644 --- a/examples/require-execution/index.html +++ b/examples/require-execution/index.html @@ -11,7 +11,7 @@ - + @@ -27,7 +27,7 @@
machinable

Appearance

On this page

Require execution ​

A way to assert that components have been cached.

Usage example ​

python
from machinable import get
 
 with get("require"):
-    ... # components to check

Source ​

py
from machinable import Execution
+    ... # components to check

Source ​

py
from machinable import Execution
 
 
 class Require(Execution):
@@ -62,7 +62,7 @@
                 )
             )
         )

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/slurm-execution/index.html b/examples/slurm-execution/index.html index 27f8682f..802d5254 100644 --- a/examples/slurm-execution/index.html +++ b/examples/slurm-execution/index.html @@ -11,7 +11,7 @@ - + @@ -27,7 +27,7 @@
machinable

Appearance

On this page

Slurm execution ​

Integration to submit to the Slurm scheduler.

Usage example ​

python
from machinable import get
 
 with get("slurm", {"ranks": 8, 'preamble': 'mpirun'}):
-    ... # your component

Source ​

py
from typing import Literal, Optional, Union
+    ... # your component

Source ​

py
from typing import Literal, Optional, Union
 
 import os
 import subprocess
@@ -385,7 +385,7 @@
                 canonicalized[prefix + "--" + k] = str(v)
 
         return canonicalized

MIT Licensed

Copyright © 2021-present

- diff --git a/guide/component.html b/guide/component.html index 5657d9a3..0b7afde7 100644 --- a/guide/component.html +++ b/guide/component.html @@ -11,7 +11,7 @@ - + @@ -43,7 +43,7 @@ >>> mnist.launch() Downloading 'mnist' ...

If the execution is successful, the component is marked as finished.

python
>>> mnist.execution.is_finished()
 True

By design, component instances can only be executed once. They are automatically assigned a timestamp, random seed, as well as a nickname for easy identification.

python
>>> mnist.seed
-1632827863

Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.

Implementing custom execution ​

Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.

To implement an execution, implement an interface that inherits from the Execution base class, for example:

python
from multiprocessing import Pool
+1632827863

Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.

Implementing custom execution ​

Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.

To implement an execution, implement an interface that inherits from the Execution base class, for example:

python
from multiprocessing import Pool
 
 from machinable import Execution
 
@@ -66,7 +66,7 @@
 multiprocess = get("multiprocess", {'processes': 2})

Then, to use it, we can wrap the launch in the execution context:

python
with multiprocessing:
     mnist.launch()

Check out the execution examples that include generally useful implementations you may like to use in your projects.

Using the CLI ​

Components can be launched directly from the command-line. The CLI works out of the box and closely mirrors the Python interface. To run a component, type its module name and method name, optionally followed by the configuration options, for example:

bash
machinable mnist_data batch_size=4 --launch

To use multiprocessing, you may type:

bash
machinable mnist_data batch_size=4 \
   multiprocess processes=4 --launch

MIT Licensed

Copyright © 2021-present

- diff --git a/guide/element.html b/guide/element.html index b77f403d..61196f13 100644 --- a/guide/element.html +++ b/guide/element.html @@ -78,7 +78,7 @@ {'batch_size': 8, 'name': 'mnist'} >>> mnist.config._update_ {'batch_size': 1}

Elements support many more advanced configuration features such as typing, validation, parameter documentation, computed values, etc., which will be covered in later sections of the Guide. For now, to summarize, elements are classes with default configurations that may be modified with a list of configuration updates.

- diff --git a/guide/installation.html b/guide/installation.html index b808dec5..66c775ef 100644 --- a/guide/installation.html +++ b/guide/installation.html @@ -25,7 +25,7 @@
machinable

Appearance

On this page

Installation ​

machinable is available via pip. Install the current release

bash
$ pip install machinable

INFO

machinable currently supports Python 3.8 and higher

Note that machinable requires the sqlite json1 extension, otherwise, you will likely see the error message: sqlite3.OperationalError: no such function: json_extract. In this case, an easy way to obtain a suitable sqlite version is to install the pysqlite package:

bash
$ pip install pysqlite3-binary

MIT Licensed

Copyright © 2021-present

- diff --git a/guide/interface.html b/guide/interface.html index d9ed1da7..3d0ddcbe 100644 --- a/guide/interface.html +++ b/guide/interface.html @@ -11,7 +11,7 @@ - + @@ -47,7 +47,7 @@ ├─ estimate_gravity.py # contains a data analysis component ├─ evolution/ | └─ simulate_offspring.py # contains a evolutionary simulation -└─ main.py # main script to execute

The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:

python
from machinable import get
+└─ main.py                        # main script to execute

The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:

python
from machinable import get
 
 from estimate_gravity import EstimateGravity
 from evolution.simulate_offspring import SimulateOffspring
@@ -67,7 +67,7 @@
 with open(mnist.local_directory("download_script.sh"), "w") as f:
     f.write(...)
     os.chmod(f.name, 0o755)

Overall, interfaces make it easy to associate data with code as instantiation, storage and retrieval are managed automatically behind the scenes.

- diff --git a/guide/introduction.html b/guide/introduction.html index d9f4ab2b..9f9ea945 100644 --- a/guide/introduction.html +++ b/guide/introduction.html @@ -11,7 +11,7 @@ - + @@ -24,7 +24,7 @@ -
machinable

Appearance

On this page

Introduction ​

What is machinable? ​

machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.

The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.

  1. Write some code
python
from random import random
+    
machinable
Appearance
On this page
Introduction ​
What is machinable? ​
machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.
The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.
  1. Write some code
python
from random import random
 
 from pydantic import BaseModel, Field
 
@@ -52,7 +52,7 @@
             print(
                 f"After {self.config.samples} samples, "
                 f"PI is approximately {self.load_file('result.json')['pi']}."
-            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
+            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
 
 # Imports component in `montecarlo.py` with samples=150;
 # if an component with this configuration exists, it
@@ -76,7 +76,7 @@
 >>> experiment.local_directory()
 './storage/24aee0fd05024400b116593d1436e9f5'
bash
$ machinable montecarlo samples=150 --launch --summary
 > After 150 samples, PI is approximately 3.1466666666666665.
The above example demonstrates the two core principles of machinable code:
  • Enforced modularity The Monte Carlo algorithm is encapsulated in its own module that can be instantiated with different configuration settings.
  • Unified representation Running code is handled through the same interface that is used to retrieve produced results; multiple invocations simply reload and display the results without re-running the experiment.
You may already have questions - don't worry. We will cover the details in the rest of the documentation. For now, please read along so you can have a high-level understanding of what machinable offers.
What it is not ​
Research is extremely diverse so machinable primarily aims to be an API-spec that leaves concrete feature implementation to the user. Check out the examples to learn what this looks like in practice.
Where to go from here ​
⚙️   Installation
We recommend installing machinable to try things out while following along.
🧑‍🎓   Continue with the Guide
Designed to learn concepts hands-on. Starts with the bare minimum of concepts necessary to start using machinable. Along the way, it will provide pointers to sections that discuss concepts in more detail or cover more advanced functionality.
➡️   Check out the How-to guides
Explore real-world examples that demonstrate advanced concepts
📖   Consult the Reference
Describes available APIs in full detail.
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/hashmap.json b/hashmap.json
index 479a1a72..3db95aa3 100644
--- a/hashmap.json
+++ b/hashmap.json
@@ -1 +1 @@
-{"examples_index.md":"6a333571","examples_dependent-schedules_index.md":"150d073c","examples_aimstack-storage_index.md":"6eacb297","guide_interface.md":"8d3c5449","index.md":"ec5fddb7","about_approach.md":"40736853","examples_slurm-execution_index.md":"939e1383","examples_require-execution_index.md":"08f79fdd","guide_introduction.md":"56946a6b","examples_globus-storage_index.md":"f1237033","guide_component.md":"eef0762f","guide_installation.md":"59aa7ee4","reference_index.md":"a073225b","guide_element.md":"b35f8b9f","examples_mpi-execution_index.md":"bf53f2d6"}
+{"guide_installation.md":"59aa7ee4","examples_dependent-schedules_index.md":"150d073c","index.md":"ec5fddb7","about_approach.md":"40736853","examples_index.md":"6a333571","guide_interface.md":"03f9e498","guide_element.md":"b35f8b9f","guide_component.md":"d7585b40","examples_require-execution_index.md":"5e46839f","examples_aimstack-storage_index.md":"c3418677","reference_index.md":"a073225b","examples_mpi-execution_index.md":"9dd8e138","guide_introduction.md":"619bab14","examples_slurm-execution_index.md":"68c750f8","examples_globus-storage_index.md":"a8bc1101"}
diff --git a/index.html b/index.html
index 7bbb9ead..80699d1f 100644
--- a/index.html
+++ b/index.html
@@ -25,7 +25,7 @@
   
   
     
machinable
Appearance
machinable
research code
A modular system to manage research code so you can move quickly while enabling reuse and collaboration.
Get Started
View on GitHub
machinable-logo
🛠️
Unified representation
Run code and inspect results using the same abstraction. Check out the example below ⏬
⚡️
Designed for rapid iteration
Spend more time experimenting while relying on machinable to keep things organized.
💡
Hackable and interactive
Tweak, extend, override while leveraging first-class support for Jupyter as well as the CLI.




   💻 
Some research code
Running code ...
python regression.py --rate=0.1 --logs=1 --name=run-01
... and loading the corresponding results ...
python plot_regression_result.py --component=run-01
... are distinct and often redundant.
This means you have to manually keep track by remembering what the component with rate=0.1 was called.

machinable research code
Running code ...
machinable regression rate=0.1 logs_=True --launch
... and loading the corresponding results ...
machinable regression rate=0.1 logs_=True --launch --plot
... are distinct but use the same abstraction.
This means no need to worry about names as machinable automatically keeps track if you ran rate=0.1 before.

➡️ Learn more about machinable's approach



logo
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/reference/index.html b/reference/index.html
index e84fad30..c0343b7b 100644
--- a/reference/index.html
+++ b/reference/index.html
@@ -25,7 +25,7 @@
   
   
     
machinable
Appearance
On this page
Reference documentation ​
This page contains detailed API reference documentation. It is intended to be an in-depth resource for understanding the implementation details of machinable's interfaces. You may prefer reviewing the more explanatory guide before consulting this reference.
API ​
MIT Licensed
Copyright © 2021-present

-