Skip to content

Commit

Permalink
Deployed f6ca9b2 with MkDocs version: 1.0.4
Browse files Browse the repository at this point in the history
  • Loading branch information
ncrocfer committed Oct 16, 2020
1 parent 626ad91 commit a09dfe1
Show file tree
Hide file tree
Showing 10 changed files with 160 additions and 47 deletions.
12 changes: 6 additions & 6 deletions guides/build-workflows/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,7 @@ <h1 id="build-workflows">Build Workflows</h1>
<p>Director separates the tasks logic from the workflows definition by providing a
simple YAML syntax.</p>
<p>Let's imagine the following tasks :</p>
<div class="python codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<span class="kn">from</span> <span class="nn">director</span> <span class="kn">import</span> <span class="n">task</span>

<span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;A&quot;</span><span class="p">)</span>
Expand All @@ -498,7 +498,7 @@ <h1 id="build-workflows">Build Workflows</h1>

<h2 id="chaining-multiple-tasks">Chaining multiple tasks</h2>
<p>Chaining these tasks in the <code>workflows.yml</code> file is pretty simple :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="c1"># Chain example</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># Chain example</span>
<span class="c1">#</span>
<span class="c1"># +-------+ +-------+ +-------+</span>
<span class="c1"># | A +-----&gt;+ B +-----&gt;+ C |</span>
Expand All @@ -516,7 +516,7 @@ <h2 id="chaining-multiple-tasks">Chaining multiple tasks</h2>
<h2 id="launch-tasks-in-parallel">Launch tasks in parallel</h2>
<p>Sometimes you need to execute some tasks in parallel to improve your workflow performance.
The <code>type: group</code> keywords can be used to handle this canvas :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="c1"># Group example</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># Group example</span>
<span class="c1"># +-------+</span>
<span class="c1"># +--&gt;+ B |</span>
<span class="c1"># +-------+ | +-------+</span>
Expand All @@ -540,7 +540,7 @@ <h2 id="periodic-workflows">Periodic workflows</h2>
<p>Celery provides a scheduler used to periodically execute some tasks. This scheduler is named
the <a href="https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html">Celery beat</a>.</p>
<p>Director allows you to periodically schedule a whole workflow using a simple YAML syntax :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="nt">example.CHAIN</span><span class="p">:</span>
<div class="codehilite"><pre><span></span><code><span class="nt">example.CHAIN</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">A</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">B</span>
Expand All @@ -565,7 +565,7 @@ <h2 id="periodic-workflows">Periodic workflows</h2>
</div>
<h2 id="use-of-queues-in-workflows">Use of queues in Workflows</h2>
<p>With director, you can set queues for workflows. All workflow's tasks will use the same queue:</p>
<div class="yaml codehilite"><pre><span></span><code><span class="nt">example.ETL</span><span class="p">:</span>
<div class="codehilite"><pre><span></span><code><span class="nt">example.ETL</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">A</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">B</span>
Expand All @@ -574,7 +574,7 @@ <h2 id="use-of-queues-in-workflows">Use of queues in Workflows</h2>
</code></pre></div>

<p>You need the start Celery worker instance with the <code>--queues</code> option:</p>
<div class="bash codehilite"><pre><span></span><code>$ director celery worker --loglevel<span class="o">=</span>INFO --queues<span class="o">=</span>q1
<div class="codehilite"><pre><span></span><code>$ director celery worker --loglevel<span class="o">=</span>INFO --queues<span class="o">=</span>q1
</code></pre></div>


Expand Down
4 changes: 2 additions & 2 deletions guides/enable-authentication/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -438,11 +438,11 @@ <h1 id="enable-authentication">Enable authentication</h1>
<p>Director provide basic authentication. To enable it, you have to create users and set <code>DIRECTOR_AUTH_ENABLED</code> variable to <strong>true</strong> in the <code>.env</code> file.</p>
<h2 id="manage-user">Manage user</h2>
<p>You can manage users using the CLI.</p>
<div class="bash codehilite"><pre><span></span><code>$ director user <span class="o">[</span>create<span class="p">|</span>list<span class="p">|</span>update<span class="p">|</span>delete<span class="o">]</span>
<div class="codehilite"><pre><span></span><code>$ director user <span class="o">[</span>create<span class="p">|</span>list<span class="p">|</span>update<span class="p">|</span>delete<span class="o">]</span>
</code></pre></div>

<p>Create user example:</p>
<div class="bash codehilite"><pre><span></span><code>$ director user create john
<div class="codehilite"><pre><span></span><code>$ director user create john
</code></pre></div>


Expand Down
6 changes: 3 additions & 3 deletions guides/error-tracking/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -392,11 +392,11 @@
<h1 id="error-tracking">Error Tracking</h1>
<p>Director can send errors to <a href="https://docs.sentry.io/">Sentry</a>.</p>
<p>You can enable this feature by adding your <code>SENTRY_DSN</code> value in the <code>DIRECTOR_SENTRY_DSN</code> variable of the <code>.env</code> file :</p>
<div class="bash codehilite"><pre><span></span><code><span class="nv">DIRECTOR_SENTRY_DSN</span><span class="o">=</span><span class="s2">&quot;https://xyz@sentry.example.com/0&quot;</span>
<div class="codehilite"><pre><span></span><code><span class="nv">DIRECTOR_SENTRY_DSN</span><span class="o">=</span><span class="s2">&quot;https://xyz@sentry.example.com/0&quot;</span>
</code></pre></div>

<p>Let's imagine the following workflow :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="c1"># workflows.yml</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># workflows.yml</span>
<span class="nn">---</span>
<span class="nt">demo.SENTRY_ALERT</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
Expand All @@ -405,7 +405,7 @@ <h1 id="error-tracking">Error Tracking</h1>
</code></pre></div>

<p>With the associated tasks :</p>
<div class="python codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<span class="kn">from</span> <span class="nn">director</span> <span class="kn">import</span> <span class="n">task</span>

<span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;WORKING_TASK&quot;</span><span class="p">)</span>
Expand Down
12 changes: 6 additions & 6 deletions guides/use-payload/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,7 @@ <h1 id="use-payload">Use Payload</h1>
<p>Most of the time your workflow and their tasks will not be <em>static</em> but will
depend on some payload to work.</p>
<p>For example you can have the following workflow :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="nt">product.ORDER</span><span class="p">:</span>
<div class="codehilite"><pre><span></span><code><span class="nt">product.ORDER</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">ORDER_PRODUCT</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">SEND_MAIL</span>
Expand All @@ -506,7 +506,7 @@ <h2 id="send-payload">Send payload</h2>

<h2 id="handle-payload">Handle payload</h2>
<p>You can handle the payload in the code using the <strong>kwargs</strong> dictionnary :</p>
<div class="python codehilite"><pre><span></span><code><span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;ORDER_PRODUCT&quot;</span><span class="p">)</span>
<div class="codehilite"><pre><span></span><code><span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;ORDER_PRODUCT&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">order_product</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">order</span> <span class="o">=</span> <span class="n">Order</span><span class="p">(</span>
<span class="n">user</span><span class="o">=</span><span class="n">kwargs</span><span class="p">[</span><span class="s2">&quot;payload&quot;</span><span class="p">][</span><span class="s2">&quot;user&quot;</span><span class="p">],</span>
Expand Down Expand Up @@ -543,7 +543,7 @@ <h2 id="create-the-schema">Create the schema</h2>
</code></pre></div>

<p>Then you can reference it in your workflow using the <code>schema</code> keyword :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="nt">product.ORDER</span><span class="p">:</span>
<div class="codehilite"><pre><span></span><code><span class="nt">product.ORDER</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">ORDER_PRODUCT</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">SEND_MAIL</span>
Expand All @@ -556,7 +556,7 @@ <h2 id="create-the-schema">Create the schema</h2>
and reference it in your YAML file with : <code>schema: foo/bar/baz</code>.</p>
</div>
<p>From now the execution will be blocked if the payload is not valid :</p>
<div class="text codehilite"><pre><span></span><code>$ director workflow run product.ORDER &#39;{&quot;user&quot;: &quot;john&quot;, &quot;product&quot;: 1000}&#39;
<div class="codehilite"><pre><span></span><code>$ director workflow run product.ORDER &#39;{&quot;user&quot;: &quot;john&quot;, &quot;product&quot;: 1000}&#39;
Error: Payload is not valid
- &#39;john&#39; is not of type &#39;integer&#39;
Aborted!
Expand All @@ -566,7 +566,7 @@ <h2 id="create-the-schema">Create the schema</h2>
<h2 id="periodic-workflows">Periodic workflows</h2>
<p>Celery Director provides a YAML syntax to <a href="../build-workflows/#periodic-workflows">periodically schedule a workflow</a>.
If your workflow needs a payload to work, you can send it default values :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="nt">users.UPDATE_CACHE</span><span class="p">:</span>
<div class="codehilite"><pre><span></span><code><span class="nt">users.UPDATE_CACHE</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">UPDATE_CACHE</span>
<span class="nt">periodic</span><span class="p">:</span>
Expand All @@ -575,7 +575,7 @@ <h2 id="periodic-workflows">Periodic workflows</h2>
</code></pre></div>

<p>The corresponding task can easily handle this default value :</p>
<div class="python codehilite"><pre><span></span><code><span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;UPDATE_CACHE&quot;</span><span class="p">)</span>
<div class="codehilite"><pre><span></span><code><span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;UPDATE_CACHE&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">update_cache</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">kwargs</span><span class="p">[</span><span class="s2">&quot;payload&quot;</span><span class="p">][</span><span class="s2">&quot;user&quot;</span><span class="p">]</span>

Expand Down
8 changes: 4 additions & 4 deletions guides/write-tasks/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -484,7 +484,7 @@ <h2 id="create-a-task">Create a task</h2>
the <code>app.task()</code> decorator to transform Python function into Celery tasks.</p>
<p>This work has already be done for you in Director, so you just have to transform your function
using the <code>director.task</code> decorator :</p>
<div class="python codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<span class="kn">from</span> <span class="nn">director</span> <span class="kn">import</span> <span class="n">task</span>


Expand Down Expand Up @@ -514,7 +514,7 @@ <h2 id="task-signature">Task signature</h2>
parents in the <code>args</code> parameter we always use the <code>s()</code> method.</p>
</div>
<p>Here is are some concrete examples based on the following tasks :</p>
<div class="python codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<span class="kn">from</span> <span class="nn">director</span> <span class="kn">import</span> <span class="n">task</span>

<span class="nd">@task</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;A&quot;</span><span class="p">)</span>
Expand All @@ -532,7 +532,7 @@ <h2 id="task-signature">Task signature</h2>

<p>The following workflows present different usecases and the output of the C task (see the
<a href="../build-workflows/">Build Workflows</a> guide to understand the YAML format) :</p>
<div class="yaml codehilite"><pre><span></span><code><span class="nt">example.NO_PARENT</span><span class="p">:</span>
<div class="codehilite"><pre><span></span><code><span class="nt">example.NO_PARENT</span><span class="p">:</span>
<span class="nt">tasks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">C</span>
<span class="c1"># Result : (None,)</span>
Expand All @@ -559,7 +559,7 @@ <h2 id="bound-tasks">Bound Tasks</h2>
<p>Celery allows use to <a href="https://docs.celeryproject.org/en/latest/userguide/tasks.html#bound-tasks">bind a task</a>,
providing the task instance itself as the first parameter.</p>
<p>In this case the signature will must contain a first parameter just before <code>args</code> and <code>kwargs</code> :</p>
<div class="python codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<div class="codehilite"><pre><span></span><code><span class="c1"># tasks/example.py</span>
<span class="kn">from</span> <span class="nn">director</span> <span class="kn">import</span> <span class="n">task</span>

<span class="nd">@task</span><span class="p">(</span><span class="n">bind</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">&quot;BOUND_TASK&quot;</span><span class="p">)</span>
Expand Down

0 comments on commit a09dfe1

Please sign in to comment.