Permalink
Browse files

update docs

  • Loading branch information...
1 parent 122cb81 commit 970d65caa3df621f147272e146eb578593e57e29 @lirazsiri lirazsiri committed Aug 3, 2011
Showing with 120 additions and 108 deletions.
  1. +37 −33 docs/cloudtask.html
  2. +46 −42 docs/cloudtask.man
  3. +37 −33 docs/cloudtask.txt
View
70 docs/cloudtask.html
@@ -440,17 +440,17 @@ <h2 class="subtitle" id="parallel-batch-execution-with-auto-launched-cloud-serve
launching and destroying server instances reliably on demand and
distributing work amongst them is required.</p>
<p>CloudTask solves this problem by automating the execution of a batch job
-on remote worker servers over SSH. The user can split up the batch
-amongst an arbitrary number of servers to speed up execution time.
-CloudTask can automatically allocate (and later destroy) EC2 cloud
-servers as required, or the user may provide a list of suitably
-configured &quot;persistent&quot; servers.</p>
+on remote worker servers over SSH. The user can split up the batch so
+that its parts run in parallel amongst an arbitrary number of servers to
+speed up execution time. CloudTask can automatically allocate (and
+later destroy) EC2 cloud servers as required, or the user may provide a
+list of suitably configured &quot;persistent&quot; servers.</p>
</div>
<div class="section" id="terms-and-definitions">
<h2>Terms and definitions</h2>
<ul class="simple">
-<li>Task: a sequence of jobs</li>
<li>Job: a shell command representing an atomic unit of work</li>
+<li>Task: a sequence of jobs</li>
<li>Task template: a pre-configured task.</li>
<li>Session: the state of a task run at a particular time. This includes
the task configuration, the status of jobs that have finished
@@ -570,6 +570,7 @@ <h2 class="subtitle" id="parallel-batch-execution-with-auto-launched-cloud-serve
class RefreshISO(Task):
DESCRIPTION = &quot;This task refreshes security updates on an ISO&quot;
+ BACKUP_ID = 123
COMMAND = 'refresh-iso-security-updates'
SPLIT = 10
REPORT = 'mail: cloudtask&#64;example.com alon&#64;example.com liraz&#64;example.com'
@@ -595,6 +596,7 @@ <h2 class="subtitle" id="parallel-batch-execution-with-auto-launched-cloud-serve
ec2-size m1.small
ec2-type s3
user root
+ backup-id 123
workers -
overlay -
post -
@@ -623,8 +625,27 @@ <h2 class="subtitle" id="parallel-batch-execution-with-auto-launched-cloud-serve
</pre>
</div>
</div>
-<div class="section" id="configuration">
-<h1>CONFIGURATION</h1>
+<div class="section" id="getting-started">
+<h1>GETTING STARTED</h1>
+<p>Since launching and destroying cloud servers can take a few minutes, the
+easiest way to get started and explore cloudtask is to experiment with a
+local ssh server:</p>
+<pre class="literal-block">
+# you need root privileges to install SSH
+apt-get install openssh-server
+/etc/init.d/ssh start
+</pre>
+<p>Add your user's SSH key to root's authorized keys:</p>
+<pre class="literal-block">
+ssh-copy-id root&#64;localhost
+</pre>
+<p>Then run test tasks with the --workers=localhost option, like this:</p>
+<pre class="literal-block">
+seq 10 | cloudtask --workers=localhost echo
+</pre>
+</div>
+<div class="section" id="task-configuration">
+<h1>TASK CONFIGURATION</h1>
<p>Any cloudtask configuration option that can be configured from the
command line may also be configured through a template default, or by
defining an environment variable.</p>
@@ -662,32 +683,12 @@ <h2 class="subtitle" id="parallel-batch-execution-with-auto-launched-cloud-serve
</pre>
</li>
</ol>
-</div>
-<div class="section" id="exploring-cloudtask">
-<h1>EXPLORING CLOUDTASK</h1>
-<p>Since launching and destroying cloud servers can take a few minutes, the
-easiest way to explore and experiment with cloudtask is to run a local
-ssh server:</p>
-<pre class="literal-block">
-# you need root privileges to install SSH
-apt-get install openssh-server
-/etc/init.d/ssh start
-</pre>
-<p>Add your user's SSH key to root's authorized keys:</p>
-<pre class="literal-block">
-ssh-copy-id root&#64;localhost
-</pre>
-<p>Then run test tasks with the --workers=localhost option, like this:</p>
-<pre class="literal-block">
-seq 10 | cloudtask --workers=localhost echo
-</pre>
-</div>
<div class="section" id="best-practices-for-production-use">
-<h1>BEST PRACTICES FOR PRODUCTION USE</h1>
-<p>For production use, it is recommended to create pre-configured templates
-for routine jobs in a Git repository. Templates may inherit shared
-definitions such as the Hub APIKEY or the reporting hook from a common
-module:</p>
+<h2>BEST PRACTICES FOR PRODUCTION USE</h2>
+<p>For production use, it is recommended to create pre-configured task
+templates for routine jobs in a Git repository. Task templates may
+inherit shared definitions such as the Hub APIKEY or the reporting hook
+from a common module:</p>
<pre class="literal-block">
$ cat &gt; common.py &lt;&lt; 'EOF'
from cloudtask import Task
@@ -702,14 +703,17 @@ <h2 class="subtitle" id="parallel-batch-execution-with-auto-launched-cloud-serve
EOF
$ cat &gt; helloworld &lt;&lt; 'EOF'
+#!/usr/bin/python
from common import BaseTask
class HelloWorld(BaseTask):
COMMAND = 'echo hello world'
HelloWorld.main()
EOF
+chmod +x helloworld
</pre>
</div>
+</div>
<div class="section" id="how-it-works">
<h1>HOW IT WORKS</h1>
<p>When the user executes a task, the following steps are performed:</p>
View
88 docs/cloudtask.man
@@ -153,19 +153,19 @@ launching and destroying server instances reliably on demand and
distributing work amongst them is required.
.sp
CloudTask solves this problem by automating the execution of a batch job
-on remote worker servers over SSH. The user can split up the batch
-amongst an arbitrary number of servers to speed up execution time.
-CloudTask can automatically allocate (and later destroy) EC2 cloud
-servers as required, or the user may provide a list of suitably
-configured "persistent" servers.
+on remote worker servers over SSH. The user can split up the batch so
+that its parts run in parallel amongst an arbitrary number of servers to
+speed up execution time. CloudTask can automatically allocate (and
+later destroy) EC2 cloud servers as required, or the user may provide a
+list of suitably configured "persistent" servers.
.SS Terms and definitions
.INDENT 0.0
.IP \(bu 2
.
-Task: a sequence of jobs
+Job: a shell command representing an atomic unit of work
.IP \(bu 2
.
-Job: a shell command representing an atomic unit of work
+Task: a sequence of jobs
.IP \(bu 2
.
Task template: a pre\-configured task.
@@ -328,6 +328,7 @@ from cloudtask import Task
class RefreshISO(Task):
DESCRIPTION = "This task refreshes security updates on an ISO"
+ BACKUP_ID = 123
COMMAND = \(aqrefresh\-iso\-security\-updates\(aq
SPLIT = 10
REPORT = \(aqmail: cloudtask@example.com alon@example.com liraz@example.com\(aq
@@ -353,6 +354,7 @@ About to launch 10 cloud servers to execute the following task:
ec2\-size m1.small
ec2\-type s3
user root
+ backup\-id 123
workers \-
overlay \-
post \-
@@ -385,7 +387,36 @@ jobs in real\-time by tailing the worker log files:
tail \-f ~/.cloudtask/11/workers/29721
.ft P
.fi
-.SH CONFIGURATION
+.SH GETTING STARTED
+.sp
+Since launching and destroying cloud servers can take a few minutes, the
+easiest way to get started and explore cloudtask is to experiment with a
+local ssh server:
+.sp
+.nf
+.ft C
+# you need root privileges to install SSH
+apt\-get install openssh\-server
+/etc/init.d/ssh start
+.ft P
+.fi
+.sp
+Add your user\(aqs SSH key to root\(aqs authorized keys:
+.sp
+.nf
+.ft C
+ssh\-copy\-id root@localhost
+.ft P
+.fi
+.sp
+Then run test tasks with the \-\-workers=localhost option, like this:
+.sp
+.nf
+.ft C
+seq 10 | cloudtask \-\-workers=localhost echo
+.ft P
+.fi
+.SH TASK CONFIGURATION
.sp
Any cloudtask configuration option that can be configured from the
command line may also be configured through a template default, or by
@@ -438,41 +469,12 @@ export CLOUDTASK_EC2_REGION=ap\-southeast\-1
.ft P
.fi
.UNINDENT
-.SH EXPLORING CLOUDTASK
-.sp
-Since launching and destroying cloud servers can take a few minutes, the
-easiest way to explore and experiment with cloudtask is to run a local
-ssh server:
-.sp
-.nf
-.ft C
-# you need root privileges to install SSH
-apt\-get install openssh\-server
-/etc/init.d/ssh start
-.ft P
-.fi
-.sp
-Add your user\(aqs SSH key to root\(aqs authorized keys:
-.sp
-.nf
-.ft C
-ssh\-copy\-id root@localhost
-.ft P
-.fi
-.sp
-Then run test tasks with the \-\-workers=localhost option, like this:
+.SS BEST PRACTICES FOR PRODUCTION USE
.sp
-.nf
-.ft C
-seq 10 | cloudtask \-\-workers=localhost echo
-.ft P
-.fi
-.SH BEST PRACTICES FOR PRODUCTION USE
-.sp
-For production use, it is recommended to create pre\-configured templates
-for routine jobs in a Git repository. Templates may inherit shared
-definitions such as the Hub APIKEY or the reporting hook from a common
-module:
+For production use, it is recommended to create pre\-configured task
+templates for routine jobs in a Git repository. Task templates may
+inherit shared definitions such as the Hub APIKEY or the reporting hook
+from a common module:
.sp
.nf
.ft C
@@ -489,12 +491,14 @@ class BaseTask(Task):
EOF
$ cat > helloworld << \(aqEOF\(aq
+#!/usr/bin/python
from common import BaseTask
class HelloWorld(BaseTask):
COMMAND = \(aqecho hello world\(aq
HelloWorld.main()
EOF
+chmod +x helloworld
.ft P
.fi
.SH HOW IT WORKS
View
70 docs/cloudtask.txt
@@ -110,19 +110,19 @@ launching and destroying server instances reliably on demand and
distributing work amongst them is required.
CloudTask solves this problem by automating the execution of a batch job
-on remote worker servers over SSH. The user can split up the batch
-amongst an arbitrary number of servers to speed up execution time.
-CloudTask can automatically allocate (and later destroy) EC2 cloud
-servers as required, or the user may provide a list of suitably
-configured "persistent" servers.
+on remote worker servers over SSH. The user can split up the batch so
+that its parts run in parallel amongst an arbitrary number of servers to
+speed up execution time. CloudTask can automatically allocate (and
+later destroy) EC2 cloud servers as required, or the user may provide a
+list of suitably configured "persistent" servers.
Terms and definitions
---------------------
-* Task: a sequence of jobs
-
* Job: a shell command representing an atomic unit of work
+* Task: a sequence of jobs
+
* Task template: a pre-configured task.
* Session: the state of a task run at a particular time. This includes
@@ -249,6 +249,7 @@ creates a pre-configured cloudtask template for it in $HOME/cloudtasks::
class RefreshISO(Task):
DESCRIPTION = "This task refreshes security updates on an ISO"
+ BACKUP_ID = 123
COMMAND = 'refresh-iso-security-updates'
SPLIT = 10
REPORT = 'mail: cloudtask@example.com alon@example.com liraz@example.com'
@@ -274,6 +275,7 @@ creates a pre-configured cloudtask template for it in $HOME/cloudtasks::
ec2-size m1.small
ec2-type s3
user root
+ backup-id 123
workers -
overlay -
post -
@@ -301,8 +303,27 @@ jobs in real-time by tailing the worker log files::
tail -f ~/.cloudtask/11/workers/29721
-CONFIGURATION
-=============
+GETTING STARTED
+===============
+
+Since launching and destroying cloud servers can take a few minutes, the
+easiest way to get started and explore cloudtask is to experiment with a
+local ssh server::
+
+ # you need root privileges to install SSH
+ apt-get install openssh-server
+ /etc/init.d/ssh start
+
+Add your user's SSH key to root's authorized keys::
+
+ ssh-copy-id root@localhost
+
+Then run test tasks with the --workers=localhost option, like this::
+
+ seq 10 | cloudtask --workers=localhost echo
+
+TASK CONFIGURATION
+==================
Any cloudtask configuration option that can be configured from the
command line may also be configured through a template default, or by
@@ -339,32 +360,13 @@ are launched in, you can configure it as:
export CLOUDTASK_EC2_REGION=ap-southeast-1
-EXPLORING CLOUDTASK
-===================
-
-Since launching and destroying cloud servers can take a few minutes, the
-easiest way to explore and experiment with cloudtask is to run a local
-ssh server::
-
- # you need root privileges to install SSH
- apt-get install openssh-server
- /etc/init.d/ssh start
-
-Add your user's SSH key to root's authorized keys::
-
- ssh-copy-id root@localhost
-
-Then run test tasks with the --workers=localhost option, like this::
-
- seq 10 | cloudtask --workers=localhost echo
-
BEST PRACTICES FOR PRODUCTION USE
-=================================
+---------------------------------
-For production use, it is recommended to create pre-configured templates
-for routine jobs in a Git repository. Templates may inherit shared
-definitions such as the Hub APIKEY or the reporting hook from a common
-module::
+For production use, it is recommended to create pre-configured task
+templates for routine jobs in a Git repository. Task templates may
+inherit shared definitions such as the Hub APIKEY or the reporting hook
+from a common module::
$ cat > common.py << 'EOF'
from cloudtask import Task
@@ -379,12 +381,14 @@ module::
EOF
$ cat > helloworld << 'EOF'
+ #!/usr/bin/python
from common import BaseTask
class HelloWorld(BaseTask):
COMMAND = 'echo hello world'
HelloWorld.main()
EOF
+ chmod +x helloworld
HOW IT WORKS
============

0 comments on commit 970d65c

Please sign in to comment.