From 880222f89ca3d6b71a86fbb882eb38f4c1d8980c Mon Sep 17 00:00:00 2001 From: frthjf Date: Sat, 25 May 2024 01:34:56 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20machinab?= =?UTF-8?q?le-org/machinable@a30841c82bb4d8c7e28cef0d369ce3520083b38c=20?= =?UTF-8?q?=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- 404.html | 2 +- about/approach.html | 2 +- ....js => examples_aimstack-storage_index.md.21cd21cf.js} | 2 +- ...> examples_aimstack-storage_index.md.21cd21cf.lean.js} | 0 ...01.js => examples_globus-storage_index.md.0a7a0528.js} | 2 +- ... => examples_globus-storage_index.md.0a7a0528.lean.js} | 0 ...138.js => examples_mpi-execution_index.md.ce8c196a.js} | 2 +- ...s => examples_mpi-execution_index.md.ce8c196a.lean.js} | 0 ...js => examples_require-execution_index.md.9c5769eb.js} | 2 +- ... examples_require-execution_index.md.9c5769eb.lean.js} | 0 ...8.js => examples_slurm-execution_index.md.e6a029a3.js} | 2 +- ...=> examples_slurm-execution_index.md.e6a029a3.lean.js} | 0 ...nent.md.d7585b40.js => guide_component.md.77eaecb4.js} | 2 +- ...585b40.lean.js => guide_component.md.77eaecb4.lean.js} | 0 ...face.md.03f9e498.js => guide_interface.md.e94258c0.js} | 6 +++--- ...f9e498.lean.js => guide_interface.md.e94258c0.lean.js} | 2 +- ...n.md.619bab14.js => guide_introduction.md.05acfcda.js} | 4 ++-- ...b14.lean.js => guide_introduction.md.05acfcda.lean.js} | 0 examples/aimstack-storage/index.html | 6 +++--- examples/dependent-schedules/index.html | 2 +- examples/globus-storage/index.html | 6 +++--- examples/index.html | 2 +- examples/mpi-execution/index.html | 6 +++--- examples/require-execution/index.html | 6 +++--- examples/slurm-execution/index.html | 6 +++--- guide/component.html | 6 +++--- guide/element.html | 2 +- guide/installation.html | 2 +- guide/interface.html | 6 +++--- guide/introduction.html | 8 ++++---- hashmap.json | 2 +- index.html | 2 +- reference/index.html | 2 +- 33 files changed, 46 insertions(+), 46 deletions(-) rename assets/{examples_aimstack-storage_index.md.c3418677.js => examples_aimstack-storage_index.md.21cd21cf.js} (99%) rename assets/{examples_aimstack-storage_index.md.c3418677.lean.js => examples_aimstack-storage_index.md.21cd21cf.lean.js} (100%) rename assets/{examples_globus-storage_index.md.a8bc1101.js => examples_globus-storage_index.md.0a7a0528.js} (99%) rename assets/{examples_globus-storage_index.md.a8bc1101.lean.js => examples_globus-storage_index.md.0a7a0528.lean.js} (100%) rename assets/{examples_mpi-execution_index.md.9dd8e138.js => examples_mpi-execution_index.md.ce8c196a.js} (99%) rename assets/{examples_mpi-execution_index.md.9dd8e138.lean.js => examples_mpi-execution_index.md.ce8c196a.lean.js} (100%) rename assets/{examples_require-execution_index.md.5e46839f.js => examples_require-execution_index.md.9c5769eb.js} (98%) rename assets/{examples_require-execution_index.md.5e46839f.lean.js => examples_require-execution_index.md.9c5769eb.lean.js} (100%) rename assets/{examples_slurm-execution_index.md.68c750f8.js => examples_slurm-execution_index.md.e6a029a3.js} (99%) rename assets/{examples_slurm-execution_index.md.68c750f8.lean.js => examples_slurm-execution_index.md.e6a029a3.lean.js} (100%) rename assets/{guide_component.md.d7585b40.js => guide_component.md.77eaecb4.js} (99%) rename assets/{guide_component.md.d7585b40.lean.js => guide_component.md.77eaecb4.lean.js} (100%) rename assets/{guide_interface.md.03f9e498.js => guide_interface.md.e94258c0.js} (98%) rename assets/{guide_interface.md.03f9e498.lean.js => guide_interface.md.e94258c0.lean.js} (92%) rename assets/{guide_introduction.md.619bab14.js => guide_introduction.md.05acfcda.js} (97%) rename assets/{guide_introduction.md.619bab14.lean.js => guide_introduction.md.05acfcda.lean.js} (100%) diff --git a/404.html b/404.html index a7eee853..3fdbce15 100644 --- a/404.html +++ b/404.html @@ -21,7 +21,7 @@
machinable

Appearance

404

PAGE NOT FOUND

But if you don't change your direction, and if you keep looking, you may end up where you are heading.
Take me home

MIT Licensed

Copyright © 2021-present

- diff --git a/about/approach.html b/about/approach.html index 37d8b9d2..477800f6 100644 --- a/about/approach.html +++ b/about/approach.html @@ -25,7 +25,7 @@
machinable

Appearance

On this page

About machinable's approach ​

Optional reading

This background discusses the big-picture approach. For a hands-on tutorial, refer to the guide.

Coming soon

This section is currently under construction

MIT Licensed

Copyright © 2021-present

- diff --git a/assets/examples_aimstack-storage_index.md.c3418677.js b/assets/examples_aimstack-storage_index.md.21cd21cf.js similarity index 99% rename from assets/examples_aimstack-storage_index.md.c3418677.js rename to assets/examples_aimstack-storage_index.md.21cd21cf.js index f586e155..cf83b0f8 100644 --- a/assets/examples_aimstack-storage_index.md.c3418677.js +++ b/assets/examples_aimstack-storage_index.md.21cd21cf.js @@ -2,7 +2,7 @@ import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C= get("aimstack", {"repo": "./path/to/aim-repo"}).__enter__() -# your code

Source ​

py
from typing import List, Optional
+# your code

Source ​

py
from typing import List, Optional
 
 import os
 
diff --git a/assets/examples_aimstack-storage_index.md.c3418677.lean.js b/assets/examples_aimstack-storage_index.md.21cd21cf.lean.js
similarity index 100%
rename from assets/examples_aimstack-storage_index.md.c3418677.lean.js
rename to assets/examples_aimstack-storage_index.md.21cd21cf.lean.js
diff --git a/assets/examples_globus-storage_index.md.a8bc1101.js b/assets/examples_globus-storage_index.md.0a7a0528.js
similarity index 99%
rename from assets/examples_globus-storage_index.md.a8bc1101.js
rename to assets/examples_globus-storage_index.md.0a7a0528.js
index 09876b0a..27ec8646 100644
--- a/assets/examples_globus-storage_index.md.a8bc1101.js
+++ b/assets/examples_globus-storage_index.md.0a7a0528.js
@@ -13,7 +13,7 @@ import{_ as s,o as n,c as a,O as l}from"./chunks/framework.62020867.js";const i=
 print(matches)
 
 # download from globus
-storage.download(matches[0].uuid)

Source ​

py
from typing import TYPE_CHECKING, List
+storage.download(matches[0].uuid)

Source ​

py
from typing import TYPE_CHECKING, List
 
 import os
 import time
diff --git a/assets/examples_globus-storage_index.md.a8bc1101.lean.js b/assets/examples_globus-storage_index.md.0a7a0528.lean.js
similarity index 100%
rename from assets/examples_globus-storage_index.md.a8bc1101.lean.js
rename to assets/examples_globus-storage_index.md.0a7a0528.lean.js
diff --git a/assets/examples_mpi-execution_index.md.9dd8e138.js b/assets/examples_mpi-execution_index.md.ce8c196a.js
similarity index 99%
rename from assets/examples_mpi-execution_index.md.9dd8e138.js
rename to assets/examples_mpi-execution_index.md.ce8c196a.js
index 669a3e0b..c6c684a0 100644
--- a/assets/examples_mpi-execution_index.md.9dd8e138.js
+++ b/assets/examples_mpi-execution_index.md.ce8c196a.js
@@ -1,7 +1,7 @@
 import{_ as s,o as n,c as a,O as l}from"./chunks/framework.62020867.js";const i=JSON.parse('{"title":"MPI execution","description":"","frontmatter":{},"headers":[],"relativePath":"examples/mpi-execution/index.md","filePath":"examples/mpi-execution/index.md"}'),p={name:"examples/mpi-execution/index.md"},o=l(`
MPI execution ​
Integration to launch MPI jobs.
Usage example ​
python
from machinable import get
 
 with get("mpi", {"ranks": 8}):
-    ... # your MPI ready component
Source ​
py
from typing import Literal, Optional, Union
+    ... # your MPI ready component
Source ​
py
from typing import Literal, Optional, Union
 
 import sys
 
diff --git a/assets/examples_mpi-execution_index.md.9dd8e138.lean.js b/assets/examples_mpi-execution_index.md.ce8c196a.lean.js
similarity index 100%
rename from assets/examples_mpi-execution_index.md.9dd8e138.lean.js
rename to assets/examples_mpi-execution_index.md.ce8c196a.lean.js
diff --git a/assets/examples_require-execution_index.md.5e46839f.js b/assets/examples_require-execution_index.md.9c5769eb.js
similarity index 98%
rename from assets/examples_require-execution_index.md.5e46839f.js
rename to assets/examples_require-execution_index.md.9c5769eb.js
index 011d7612..041f543c 100644
--- a/assets/examples_require-execution_index.md.5e46839f.js
+++ b/assets/examples_require-execution_index.md.9c5769eb.js
@@ -1,7 +1,7 @@
 import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=JSON.parse('{"title":"Require execution","description":"","frontmatter":{},"headers":[],"relativePath":"examples/require-execution/index.md","filePath":"examples/require-execution/index.md"}'),p={name:"examples/require-execution/index.md"},o=l(`
Require execution ​
A way to assert that components have been cached.
Usage example ​
python
from machinable import get
 
 with get("require"):
-    ... # components to check
Source ​
py
from machinable import Execution
+    ... # components to check
Source ​
py
from machinable import Execution
 
 
 class Require(Execution):
diff --git a/assets/examples_require-execution_index.md.5e46839f.lean.js b/assets/examples_require-execution_index.md.9c5769eb.lean.js
similarity index 100%
rename from assets/examples_require-execution_index.md.5e46839f.lean.js
rename to assets/examples_require-execution_index.md.9c5769eb.lean.js
diff --git a/assets/examples_slurm-execution_index.md.68c750f8.js b/assets/examples_slurm-execution_index.md.e6a029a3.js
similarity index 99%
rename from assets/examples_slurm-execution_index.md.68c750f8.js
rename to assets/examples_slurm-execution_index.md.e6a029a3.js
index f616bfee..3f83e996 100644
--- a/assets/examples_slurm-execution_index.md.68c750f8.js
+++ b/assets/examples_slurm-execution_index.md.e6a029a3.js
@@ -1,7 +1,7 @@
 import{_ as s,o as n,c as a,O as l}from"./chunks/framework.62020867.js";const i=JSON.parse('{"title":"Slurm execution","description":"","frontmatter":{},"headers":[],"relativePath":"examples/slurm-execution/index.md","filePath":"examples/slurm-execution/index.md"}'),o={name:"examples/slurm-execution/index.md"},p=l(`
Slurm execution ​
Integration to submit to the Slurm scheduler.
Usage example ​
python
from machinable import get
 
 with get("slurm", {"ranks": 8, 'preamble': 'mpirun'}):
-    ... # your component
Source ​
py
from typing import Literal, Optional, Union
+    ... # your component
Source ​
py
from typing import Literal, Optional, Union
 
 import os
 import subprocess
diff --git a/assets/examples_slurm-execution_index.md.68c750f8.lean.js b/assets/examples_slurm-execution_index.md.e6a029a3.lean.js
similarity index 100%
rename from assets/examples_slurm-execution_index.md.68c750f8.lean.js
rename to assets/examples_slurm-execution_index.md.e6a029a3.lean.js
diff --git a/assets/guide_component.md.d7585b40.js b/assets/guide_component.md.77eaecb4.js
similarity index 99%
rename from assets/guide_component.md.d7585b40.js
rename to assets/guide_component.md.77eaecb4.js
index b4ac01fa..1403adf9 100644
--- a/assets/guide_component.md.d7585b40.js
+++ b/assets/guide_component.md.77eaecb4.js
@@ -17,7 +17,7 @@ import{_ as p,D as t,o as c,c as r,z as n,a as s,G as o,B as l,O as e}from"./chu
 >>> mnist.launch()
 Downloading 'mnist' ...
If the execution is successful, the component is marked as finished.
python
>>> mnist.execution.is_finished()
 True
By design, component instances can only be executed once. They are automatically assigned a timestamp, random seed, as well as a nickname for easy identification.
python
>>> mnist.seed
-1632827863
Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.
Implementing custom execution ​
Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.
`,8),C=e(`
python
from multiprocessing import Pool
+1632827863
Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.
Implementing custom execution ​
Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.
`,8),C=e(`
python
from multiprocessing import Pool
 
 from machinable import Execution
 
diff --git a/assets/guide_component.md.d7585b40.lean.js b/assets/guide_component.md.77eaecb4.lean.js
similarity index 100%
rename from assets/guide_component.md.d7585b40.lean.js
rename to assets/guide_component.md.77eaecb4.lean.js
diff --git a/assets/guide_interface.md.03f9e498.js b/assets/guide_interface.md.e94258c0.js
similarity index 98%
rename from assets/guide_interface.md.03f9e498.js
rename to assets/guide_interface.md.e94258c0.js
index 50e13bfc..5d2426ce 100644
--- a/assets/guide_interface.md.03f9e498.js
+++ b/assets/guide_interface.md.e94258c0.js
@@ -17,11 +17,11 @@ import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chu
       # -> this is equivalent to: MnistData({"batch_size": 8})
 mnist.commit()
Now, if we later want to retrieve this instance, we can use the same code in place of a unique ID:
python
mnist_reloaded = get(MnistData, {"batch_size": 8})
 
-assert mnist == mnist_reloaded
`,3),C=a("code",null,"MnistData",-1),u=a("code",null,"batch_size",-1),h=a("code",null,"8",-1),m=e(`
The module convention ​
As your project grows, the classes that you implement should be moved into their own Python module. You are free to structure your code as you see fit but there is one hard constraint that classes must be placed in their own modules. The project source code may, for instance, be organized like this:
example_project/
+assert mnist == mnist_reloaded
`,3),C=a("code",null,"MnistData",-1),h=a("code",null,"batch_size",-1),u=a("code",null,"8",-1),m=e(`
The module convention ​
As your project grows, the classes that you implement should be moved into their own Python module. You are free to structure your code as you see fit but there is one hard constraint that classes must be placed in their own modules. The project source code may, for instance, be organized like this:
example_project/
 ├─ estimate_gravity.py            # contains a data analysis component
 ├─ evolution/                   
 |  └─ simulate_offspring.py       # contains a evolutionary simulation
-└─ main.py                        # main script to execute
The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:
python
from machinable import get
+└─ main.py                        # main script to execute
The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:
python
from machinable import get
 
 from estimate_gravity import EstimateGravity
 from evolution.simulate_offspring import SimulateOffspring
@@ -40,4 +40,4 @@ import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chu
 mnist = get("mnist_data")
 with open(mnist.local_directory("download_script.sh"), "w") as f:
     f.write(...)
-    os.chmod(f.name, 0o755)
Overall, interfaces make it easy to associate data with code as instantiation, storage and retrieval are managed automatically behind the scenes.
`,2);function _(b,v,q,E,T,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),u,s(" of "),h,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
+    os.chmod(f.name, 0o755)
Overall, interfaces make it easy to associate data with code as instantiation, storage and retrieval are managed automatically behind the scenes.
`,2);function _(b,v,T,E,q,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),h,s(" of "),u,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
diff --git a/assets/guide_interface.md.03f9e498.lean.js b/assets/guide_interface.md.e94258c0.lean.js
similarity index 92%
rename from assets/guide_interface.md.03f9e498.lean.js
rename to assets/guide_interface.md.e94258c0.lean.js
index 715e6835..e5549f26 100644
--- a/assets/guide_interface.md.03f9e498.lean.js
+++ b/assets/guide_interface.md.e94258c0.lean.js
@@ -1 +1 @@
-import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chunks/framework.62020867.js";const k=JSON.parse('{"title":"Interface","description":"","frontmatter":{},"headers":[],"relativePath":"guide/interface.md","filePath":"guide/interface.md"}'),i={name:"guide/interface.md"},y=a("h1",{id:"interface",tabindex:"-1"},[s("Interface "),a("a",{class:"header-anchor",href:"#interface","aria-label":'Permalink to "Interface"'},"​")],-1),D=a("p",null,[a("a",{href:"./element.html"},"Elements"),s(" by themselves are limited in that they are effectively stateless. You can construct and use them but any computed result or additional information will not be persisted.")],-1),F=e("",6),A=a("code",null,"MnistData()",-1),d=e("",3),C=a("code",null,"MnistData",-1),u=a("code",null,"batch_size",-1),h=a("code",null,"8",-1),m=e("",7),f=e("",1),g=e("",2);function _(b,v,q,E,T,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),u,s(" of "),h,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
+import{_ as t,D as p,o as c,c as r,z as a,a as s,G as l,B as o,O as e}from"./chunks/framework.62020867.js";const k=JSON.parse('{"title":"Interface","description":"","frontmatter":{},"headers":[],"relativePath":"guide/interface.md","filePath":"guide/interface.md"}'),i={name:"guide/interface.md"},y=a("h1",{id:"interface",tabindex:"-1"},[s("Interface "),a("a",{class:"header-anchor",href:"#interface","aria-label":'Permalink to "Interface"'},"​")],-1),D=a("p",null,[a("a",{href:"./element.html"},"Elements"),s(" by themselves are limited in that they are effectively stateless. You can construct and use them but any computed result or additional information will not be persisted.")],-1),F=e("",6),A=a("code",null,"MnistData()",-1),d=e("",3),C=a("code",null,"MnistData",-1),h=a("code",null,"batch_size",-1),u=a("code",null,"8",-1),m=e("",7),f=e("",1),g=e("",2);function _(b,v,T,E,q,w){const n=p("Pydoc");return c(),r("div",null,[y,D,a("p",null,[s("To enable storage and retrival we can use an "),l(n,null,{default:o(()=>[s("machinable.Interface")]),_:1}),s(" class.")]),F,a("p",null,[s("Specifically, to instantiate an interface (e.g. "),A,s(") we can leverage the "),l(n,null,{default:o(()=>[s("machinable.get")]),_:1}),s(" function, which takes a class as the first argument and optional constructor arguments.")]),d,a("p",null,[s("What is happening here is that "),l(n,{caption:"get()"},{default:o(()=>[s("machinable.get")]),_:1}),s(" automatically searches the storage for an interface of type "),C,s(" with a "),h,s(" of "),u,s(". If such an instance has not been committed yet (like when initially running the code), a new instance with this configuration will be returned. But if such an instance has previously been committed, it will simply be reloaded.")]),m,a("p",null,[s("While machinable automatically commits crucial information about the interface, you can use "),l(n,null,{default:o(()=>[s("machinable.Interface.save_file")]),_:1}),s(" and "),l(n,null,{default:o(()=>[s("machinable.Interface.load_file")]),_:1}),s(" to easily store and retrieve additional custom data in different file formats:")]),f,a("p",null,[s("This may be useful to save and restore some custom state of the interface. Furthermore, you are free to implement your own methods to persist data by writing and reading from the interface's "),l(n,{caption:"local_directory()"},{default:o(()=>[s("machinable.Interface.local_directory")]),_:1}),s(":")]),g])}const S=t(i,[["render",_]]);export{k as __pageData,S as default};
diff --git a/assets/guide_introduction.md.619bab14.js b/assets/guide_introduction.md.05acfcda.js
similarity index 97%
rename from assets/guide_introduction.md.619bab14.js
rename to assets/guide_introduction.md.05acfcda.js
index bdbb884c..38364173 100644
--- a/assets/guide_introduction.md.619bab14.js
+++ b/assets/guide_introduction.md.05acfcda.js
@@ -1,4 +1,4 @@
-import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=JSON.parse('{"title":"Introduction","description":"","frontmatter":{},"headers":[],"relativePath":"guide/introduction.md","filePath":"guide/introduction.md"}'),o={name:"guide/introduction.md"},p=l(`
Introduction ​
What is machinable? ​
machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.
The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.
  1. Write some code
python
from random import random
+import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=JSON.parse('{"title":"Introduction","description":"","frontmatter":{},"headers":[],"relativePath":"guide/introduction.md","filePath":"guide/introduction.md"}'),o={name:"guide/introduction.md"},p=l(`
Introduction ​
What is machinable? ​
machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.
The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.
  1. Write some code
python
from random import random
 
 from pydantic import BaseModel, Field
 
@@ -26,7 +26,7 @@ import{_ as s,o as a,c as n,O as l}from"./chunks/framework.62020867.js";const C=
             print(
                 f"After {self.config.samples} samples, "
                 f"PI is approximately {self.load_file('result.json')['pi']}."
-            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
+            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
 
 # Imports component in \`montecarlo.py\` with samples=150;
 # if an component with this configuration exists, it
diff --git a/assets/guide_introduction.md.619bab14.lean.js b/assets/guide_introduction.md.05acfcda.lean.js
similarity index 100%
rename from assets/guide_introduction.md.619bab14.lean.js
rename to assets/guide_introduction.md.05acfcda.lean.js
diff --git a/examples/aimstack-storage/index.html b/examples/aimstack-storage/index.html
index 33f4dfdb..697ddd85 100644
--- a/examples/aimstack-storage/index.html
+++ b/examples/aimstack-storage/index.html
@@ -11,7 +11,7 @@
   
   
   
-  
+  
   
   
   
@@ -28,7 +28,7 @@
 
 get("aimstack", {"repo": "./path/to/aim-repo"}).__enter__()
 
-# your code
Source ​
py
from typing import List, Optional
+# your code
Source ​
py
from typing import List, Optional
 
 import os
 
@@ -84,7 +84,7 @@
 
         for k, v in interface.__model__.model_dump().items():
             run[k] = v
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/examples/dependent-schedules/index.html b/examples/dependent-schedules/index.html
index f98b46ed..0ad87751 100644
--- a/examples/dependent-schedules/index.html
+++ b/examples/dependent-schedules/index.html
@@ -25,7 +25,7 @@
   
   
     
machinable
Appearance
On this page
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/examples/globus-storage/index.html b/examples/globus-storage/index.html
index d8bb422c..f41f816a 100644
--- a/examples/globus-storage/index.html
+++ b/examples/globus-storage/index.html
@@ -11,7 +11,7 @@
   
   
   
-  
+  
   
   
   
@@ -39,7 +39,7 @@
 print(matches)
 
 # download from globus
-storage.download(matches[0].uuid)
Source ​
py
from typing import TYPE_CHECKING, List
+storage.download(matches[0].uuid)
Source ​
py
from typing import TYPE_CHECKING, List
 
 import os
 import time
@@ -265,7 +265,7 @@
                 found.append(item["name"])
 
         return found

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/index.html b/examples/index.html index 8743a6f0..4eab4089 100644 --- a/examples/index.html +++ b/examples/index.html @@ -25,7 +25,7 @@
machinable

Appearance

On this page

How-to examples ​

This section presents code examples that demonstrate real-world usage and my be a good starting point when implementing your own projects.

Please select an example from the menu.

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/mpi-execution/index.html b/examples/mpi-execution/index.html index 38d819d5..8478e396 100644 --- a/examples/mpi-execution/index.html +++ b/examples/mpi-execution/index.html @@ -11,7 +11,7 @@ - + @@ -27,7 +27,7 @@
machinable

Appearance

On this page

MPI execution ​

Integration to launch MPI jobs.

Usage example ​

python
from machinable import get
 
 with get("mpi", {"ranks": 8}):
-    ... # your MPI ready component

Source ​

py
from typing import Literal, Optional, Union
+    ... # your MPI ready component

Source ​

py
from typing import Literal, Optional, Union
 
 import sys
 
@@ -151,7 +151,7 @@
                     raise KeyboardInterrupt(
                         "Interrupting `" + " ".join(cmd) + "`"
                     ) from _ex

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/require-execution/index.html b/examples/require-execution/index.html index c187681e..860c37fe 100644 --- a/examples/require-execution/index.html +++ b/examples/require-execution/index.html @@ -11,7 +11,7 @@ - + @@ -27,7 +27,7 @@
machinable

Appearance

On this page

Require execution ​

A way to assert that components have been cached.

Usage example ​

python
from machinable import get
 
 with get("require"):
-    ... # components to check

Source ​

py
from machinable import Execution
+    ... # components to check

Source ​

py
from machinable import Execution
 
 
 class Require(Execution):
@@ -62,7 +62,7 @@
                 )
             )
         )

MIT Licensed

Copyright © 2021-present

- diff --git a/examples/slurm-execution/index.html b/examples/slurm-execution/index.html index 802d5254..6356e852 100644 --- a/examples/slurm-execution/index.html +++ b/examples/slurm-execution/index.html @@ -11,7 +11,7 @@ - + @@ -27,7 +27,7 @@
machinable

Appearance

On this page

Slurm execution ​

Integration to submit to the Slurm scheduler.

Usage example ​

python
from machinable import get
 
 with get("slurm", {"ranks": 8, 'preamble': 'mpirun'}):
-    ... # your component

Source ​

py
from typing import Literal, Optional, Union
+    ... # your component

Source ​

py
from typing import Literal, Optional, Union
 
 import os
 import subprocess
@@ -385,7 +385,7 @@
                 canonicalized[prefix + "--" + k] = str(v)
 
         return canonicalized

MIT Licensed

Copyright © 2021-present

- diff --git a/guide/component.html b/guide/component.html index 0b7afde7..af1f0ca8 100644 --- a/guide/component.html +++ b/guide/component.html @@ -11,7 +11,7 @@ - + @@ -43,7 +43,7 @@ >>> mnist.launch() Downloading 'mnist' ...

If the execution is successful, the component is marked as finished.

python
>>> mnist.execution.is_finished()
 True

By design, component instances can only be executed once. They are automatically assigned a timestamp, random seed, as well as a nickname for easy identification.

python
>>> mnist.seed
-1632827863

Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.

Implementing custom execution ​

Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.

To implement an execution, implement an interface that inherits from the Execution base class, for example:

python
from multiprocessing import Pool
+1632827863

Invocations of launch() after successful execution, do not trigger another execution since the component is marked as cached. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.

Implementing custom execution ​

Components can be executed in different ways. You may, for example, like to run components using multiprocessing or execute in a cloud environment. However, instead of adding the execution logic directly to your component code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the component.

To implement an execution, implement an interface that inherits from the Execution base class, for example:

python
from multiprocessing import Pool
 
 from machinable import Execution
 
@@ -66,7 +66,7 @@
 multiprocess = get("multiprocess", {'processes': 2})

Then, to use it, we can wrap the launch in the execution context:

python
with multiprocessing:
     mnist.launch()

Check out the execution examples that include generally useful implementations you may like to use in your projects.

Using the CLI ​

Components can be launched directly from the command-line. The CLI works out of the box and closely mirrors the Python interface. To run a component, type its module name and method name, optionally followed by the configuration options, for example:

bash
machinable mnist_data batch_size=4 --launch

To use multiprocessing, you may type:

bash
machinable mnist_data batch_size=4 \
   multiprocess processes=4 --launch

MIT Licensed

Copyright © 2021-present

- diff --git a/guide/element.html b/guide/element.html index 61196f13..a8a422cf 100644 --- a/guide/element.html +++ b/guide/element.html @@ -78,7 +78,7 @@ {'batch_size': 8, 'name': 'mnist'} >>> mnist.config._update_ {'batch_size': 1}

Elements support many more advanced configuration features such as typing, validation, parameter documentation, computed values, etc., which will be covered in later sections of the Guide. For now, to summarize, elements are classes with default configurations that may be modified with a list of configuration updates.

- diff --git a/guide/installation.html b/guide/installation.html index 66c775ef..c9cb4683 100644 --- a/guide/installation.html +++ b/guide/installation.html @@ -25,7 +25,7 @@
machinable

Appearance

On this page

Installation ​

machinable is available via pip. Install the current release

bash
$ pip install machinable

INFO

machinable currently supports Python 3.8 and higher

Note that machinable requires the sqlite json1 extension, otherwise, you will likely see the error message: sqlite3.OperationalError: no such function: json_extract. In this case, an easy way to obtain a suitable sqlite version is to install the pysqlite package:

bash
$ pip install pysqlite3-binary

MIT Licensed

Copyright © 2021-present

- diff --git a/guide/interface.html b/guide/interface.html index 3d0ddcbe..95690e62 100644 --- a/guide/interface.html +++ b/guide/interface.html @@ -11,7 +11,7 @@ - + @@ -47,7 +47,7 @@ ├─ estimate_gravity.py # contains a data analysis component ├─ evolution/ | └─ simulate_offspring.py # contains a evolutionary simulation -└─ main.py # main script to execute

The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:

python
from machinable import get
+└─ main.py                        # main script to execute

The benefit of this requirement is that you can refer to the classes via their module import path. For example, using this module convention, you can simplify the instantiation of classes that are located in different modules:

python
from machinable import get
 
 from estimate_gravity import EstimateGravity
 from evolution.simulate_offspring import SimulateOffspring
@@ -67,7 +67,7 @@
 with open(mnist.local_directory("download_script.sh"), "w") as f:
     f.write(...)
     os.chmod(f.name, 0o755)

Overall, interfaces make it easy to associate data with code as instantiation, storage and retrieval are managed automatically behind the scenes.

- diff --git a/guide/introduction.html b/guide/introduction.html index 9f9ea945..6f6c6b90 100644 --- a/guide/introduction.html +++ b/guide/introduction.html @@ -11,7 +11,7 @@ - + @@ -24,7 +24,7 @@ -
machinable

Appearance

On this page

Introduction ​

What is machinable? ​

machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.

The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.

  1. Write some code
python
from random import random
+    
machinable
Appearance
On this page
Introduction ​
What is machinable? ​
machinable is a Python API for research code. It provides an object-oriented skeleton that helps you develop and experiment in a unified interface while handling tedious housekeeping behind the scenes.
The key idea is to unify the running of code and the retrieval of produced results in one abstraction. A detailed discussion of this approach can be found in the about section, but for now, here is a minimal example that illustrates the idea.
  1. Write some code
python
from random import random
 
 from pydantic import BaseModel, Field
 
@@ -52,7 +52,7 @@
             print(
                 f"After {self.config.samples} samples, "
                 f"PI is approximately {self.load_file('result.json')['pi']}."
-            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
+            )
  1. Run and inspect it using a unified abstraction
python
from machinable import get
 
 # Imports component in `montecarlo.py` with samples=150;
 # if an component with this configuration exists, it
@@ -76,7 +76,7 @@
 >>> experiment.local_directory()
 './storage/24aee0fd05024400b116593d1436e9f5'
bash
$ machinable montecarlo samples=150 --launch --summary
 > After 150 samples, PI is approximately 3.1466666666666665.
The above example demonstrates the two core principles of machinable code:
  • Enforced modularity The Monte Carlo algorithm is encapsulated in its own module that can be instantiated with different configuration settings.
  • Unified representation Running code is handled through the same interface that is used to retrieve produced results; multiple invocations simply reload and display the results without re-running the experiment.
You may already have questions - don't worry. We will cover the details in the rest of the documentation. For now, please read along so you can have a high-level understanding of what machinable offers.
What it is not ​
Research is extremely diverse so machinable primarily aims to be an API-spec that leaves concrete feature implementation to the user. Check out the examples to learn what this looks like in practice.
Where to go from here ​
⚙️   Installation
We recommend installing machinable to try things out while following along.
🧑‍🎓   Continue with the Guide
Designed to learn concepts hands-on. Starts with the bare minimum of concepts necessary to start using machinable. Along the way, it will provide pointers to sections that discuss concepts in more detail or cover more advanced functionality.
➡️   Check out the How-to guides
Explore real-world examples that demonstrate advanced concepts
📖   Consult the Reference
Describes available APIs in full detail.
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/hashmap.json b/hashmap.json
index 3db95aa3..3991906b 100644
--- a/hashmap.json
+++ b/hashmap.json
@@ -1 +1 @@
-{"guide_installation.md":"59aa7ee4","examples_dependent-schedules_index.md":"150d073c","index.md":"ec5fddb7","about_approach.md":"40736853","examples_index.md":"6a333571","guide_interface.md":"03f9e498","guide_element.md":"b35f8b9f","guide_component.md":"d7585b40","examples_require-execution_index.md":"5e46839f","examples_aimstack-storage_index.md":"c3418677","reference_index.md":"a073225b","examples_mpi-execution_index.md":"9dd8e138","guide_introduction.md":"619bab14","examples_slurm-execution_index.md":"68c750f8","examples_globus-storage_index.md":"a8bc1101"}
+{"about_approach.md":"40736853","index.md":"ec5fddb7","examples_aimstack-storage_index.md":"21cd21cf","examples_index.md":"6a333571","guide_introduction.md":"05acfcda","examples_require-execution_index.md":"9c5769eb","examples_dependent-schedules_index.md":"150d073c","guide_element.md":"b35f8b9f","guide_installation.md":"59aa7ee4","examples_globus-storage_index.md":"0a7a0528","guide_component.md":"77eaecb4","examples_mpi-execution_index.md":"ce8c196a","guide_interface.md":"e94258c0","reference_index.md":"a073225b","examples_slurm-execution_index.md":"e6a029a3"}
diff --git a/index.html b/index.html
index 80699d1f..e5fdea68 100644
--- a/index.html
+++ b/index.html
@@ -25,7 +25,7 @@
   
   
     
machinable
Appearance
machinable
research code
A modular system to manage research code so you can move quickly while enabling reuse and collaboration.
Get Started
View on GitHub
machinable-logo
🛠️
Unified representation
Run code and inspect results using the same abstraction. Check out the example below ⏬
⚡️
Designed for rapid iteration
Spend more time experimenting while relying on machinable to keep things organized.
💡
Hackable and interactive
Tweak, extend, override while leveraging first-class support for Jupyter as well as the CLI.




   💻 
Some research code
Running code ...
python regression.py --rate=0.1 --logs=1 --name=run-01
... and loading the corresponding results ...
python plot_regression_result.py --component=run-01
... are distinct and often redundant.
This means you have to manually keep track by remembering what the component with rate=0.1 was called.

machinable research code
Running code ...
machinable regression rate=0.1 logs_=True --launch
... and loading the corresponding results ...
machinable regression rate=0.1 logs_=True --launch --plot
... are distinct but use the same abstraction.
This means no need to worry about names as machinable automatically keeps track if you ran rate=0.1 before.

➡️ Learn more about machinable's approach



logo
MIT Licensed
Copyright © 2021-present

-    
     
   
diff --git a/reference/index.html b/reference/index.html
index c0343b7b..fe8662ff 100644
--- a/reference/index.html
+++ b/reference/index.html
@@ -25,7 +25,7 @@
   
   
     
machinable
Appearance
On this page
Reference documentation ​
This page contains detailed API reference documentation. It is intended to be an in-depth resource for understanding the implementation details of machinable's interfaces. You may prefer reviewing the more explanatory guide before consulting this reference.
API ​
MIT Licensed
Copyright © 2021-present

-