Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

good day

  • Loading branch information...
commit b0ca7a72086134769a7f05f0233e887cf6c9798e 1 parent 15fe8ff
Michiel Baird michielbaird authored
BIN  writeup/thesis_michiel/figures/user_impl2.dia
View
Binary file not shown
BIN  writeup/thesis_michiel/figures/user_impl2.dia.autosave
View
Binary file not shown
BIN  writeup/thesis_michiel/figures/user_impl2.pdf
View
Binary file not shown
107 writeup/thesis_michiel/figures/user_impl2.svg
View
@@ -0,0 +1,107 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/PR-SVG-20010719/DTD/svg10.dtd">
+<svg width="30cm" height="16cm" viewBox="26 46 590 313" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+ <g>
+ <ellipse style="fill: #ffffff" cx="80.9327" cy="87.4664" rx="48.0673" ry="28.5336"/>
+ <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="80.9327" cy="87.4664" rx="48.0673" ry="28.5336"/>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:middle;font-family:sans-serif;font-style:normal;font-weight:normal" x="80.9327" y="91.3664">
+ <tspan x="80.9327" y="91.3664">Start Task</tspan>
+ </text>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="526" y="86">
+ <tspan x="526" y="86"></tspan>
+ </text>
+ <g>
+ <line style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x1="129.983" y1="87.2644" x2="427.264" y2="86.0401"/>
+ <polygon style="fill: #000000" points="434.764,86.0092 424.785,91.0503 427.264,86.0401 424.743,81.0504 "/>
+ <polygon style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="434.764,86.0092 424.785,91.0503 427.264,86.0401 424.743,81.0504 "/>
+ </g>
+ <g>
+ <g>
+ <rect style="fill: #ffffff" x="437" y="47" width="178" height="78"/>
+ <rect style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x="437" y="47" width="178" height="78"/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="463" y="79">
+ <tspan x="463" y="79">Rsync File Transfer</tspan>
+ </text>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="453" y="105">
+ <tspan x="453" y="105">Server to Workstation</tspan>
+ </text>
+ </g>
+ <g>
+ <line style="fill: none; fill-opacity:0; stroke-width: 2; stroke-dasharray: 4; stroke: #000000" x1="526" y1="125" x2="526" y2="165.276"/>
+ <polygon style="fill: #000000" points="526,172.776 521,162.776 526,165.276 531,162.776 "/>
+ <polygon style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="526,172.776 521,162.776 526,165.276 531,162.776 "/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="406" y="155">
+ <tspan x="406" y="155">Task in progress</tspan>
+ </text>
+ <g>
+ <g>
+ <ellipse style="fill: #ffffff" cx="526" cy="207.5" rx="74" ry="31.5"/>
+ <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="526" cy="207.5" rx="74" ry="31.5"/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="476" y="203.5">
+ <tspan x="476" y="203.5">User Indicates </tspan>
+ <tspan x="476" y="219.5">Task completion</tspan>
+ </text>
+ </g>
+ <g>
+ <line style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x1="451.009" y1="207.88" x2="231.735" y2="208.99"/>
+ <polygon style="fill: #000000" points="224.235,209.028 234.21,203.977 231.735,208.99 234.26,213.977 "/>
+ <polygon style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="224.235,209.028 234.21,203.977 231.735,208.99 234.26,213.977 "/>
+ </g>
+ <g>
+ <g>
+ <rect style="fill: #ffffff" x="41" y="170" width="180" height="79"/>
+ <rect style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x="41" y="170" width="180" height="79"/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="69" y="201.5">
+ <tspan x="69" y="201.5">Rsync File Transfer</tspan>
+ </text>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="56" y="228">
+ <tspan x="56" y="228">Workstation to Server</tspan>
+ </text>
+ </g>
+ <g>
+ <line style="fill: none; fill-opacity:0; stroke-width: 2; stroke-dasharray: 4; stroke: #000000" x1="130.194" y1="249.975" x2="129.172" y2="301.267"/>
+ <polygon style="fill: #000000" points="129.023,308.766 124.223,298.668 129.172,301.267 134.221,298.867 "/>
+ <polygon style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="129.023,308.766 124.223,298.668 129.172,301.267 134.221,298.867 "/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="154" y="272">
+ <tspan x="154" y="272">Await Task verification:</tspan>
+ <tspan x="154" y="288">During this step files can again be downloaded and uploaded</tspan>
+ </text>
+ <g>
+ <g>
+ <ellipse style="fill: #ffffff" cx="128.5" cy="335" rx="58.5" ry="23"/>
+ <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="128.5" cy="335" rx="58.5" ry="23"/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="111.5" y="340">
+ <tspan x="111.5" y="340">Verify</tspan>
+ </text>
+ </g>
+ <g>
+ <polyline style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="68.9998,335 27.9791,335 27.9791,151 382.243,151 "/>
+ <polygon style="fill: #000000" points="389.743,151 379.743,156 382.243,151 379.743,146 "/>
+ <polygon style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="389.743,151 379.743,156 382.243,151 379.743,146 "/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="109.979" y="140">
+ <tspan x="109.979" y="140">Changes Required</tspan>
+ </text>
+ <g>
+ <g>
+ <ellipse style="fill: #ffffff" cx="524.002" cy="333" rx="74" ry="22"/>
+ <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="524.002" cy="333" rx="74" ry="22"/>
+ </g>
+ <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="475.002" y="333">
+ <tspan x="475.002" y="333">Signal successor </tspan>
+ <tspan x="475.002" y="349"> tasks</tspan>
+ </text>
+ </g>
+ <g>
+ <line style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x1="187.992" y1="334.699" x2="439.277" y2="333.428"/>
+ <polygon style="fill: #000000" points="446.777,333.391 436.802,338.441 439.277,333.428 436.751,328.441 "/>
+ <polygon style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" points="446.777,333.391 436.802,338.441 439.277,333.428 436.751,328.441 "/>
+ </g>
+</svg>
96 writeup/thesis_michiel/thesis.tex
View
@@ -680,11 +680,103 @@ \subsection{Implementation}
\subsubsection{Workflow Framework}
The underlying workflow system is implemented using \emph{Python}\footnote{Python: www.python.org}, within
-the Django framework.
+the Django framework. Tasks are executed asynchronously. Each job type is handled separately.
+\begin{description}
+\item[Server Job - One to One] \hfill \\
+ This job type executed a given script on all the input files. The system generates a list of
+ absolute paths and executes the script given two parameters, the input file, and the output
+ directory. This is done for all the files input files of the task.
+\item[Server Job - Many to One] \hfill \\
+ This job type executes nearly identically to the \emph{One to One} task, however instead
+ of the script being run run multiple times, it is run once, here however all input files
+ are passed in as arguments, along with the output directory. Both server jobs can use
+ any script or applications available on the system. With applications where the parameter
+ structure does not match the required utility wrapper scripts can be used.
+\item[User Job] \hfill \\
+ This type of job is split into multiple parts. Since there is a distinct separation between
+ the server and the execution of type of task, the goal is to ensure the effective communication
+ between workstations and servers. The goal is to be able to transfer large amounts of files
+ with minimal redundancy. For this reason \emph{rsync}\footnote{Rsync Incremental file
+ transfer: rsync.samba.org} is used. This only transfers files that have not been been transfered
+ yet, as such redundancy is greatly reduced, and enables efficient recovery in case of network failure.
+
+ Figure~\ref{user_task_impl2} shows the components used to execute a user task. Each of these components
+ are built using an asynchronous object. Note that the transfer components can be used at any point
+ during the task's life cycle.
+\end{description}
+\begin{figure}[!h]
+ \begin{center}
+ \includegraphics[scale=0.4]{figures/user_impl2.pdf}
+ \end{center}
+ \caption{Components that support user jobs}
+ \label{user_task_impl2}
+\end{figure}
+ Once a task is finished it passes on to finalisation. All tasks follow this model. It
+ is assumed that all the appropriate files are present on the server at this point and
+ that the task has completed successfully. At this the output directory is scanned for
+ files recursively, new files are indexed and added to the database. The new files are then
+ added to the input files of all successor tasks. If all the dependencies of a successor
+ task is met the task is then launched.
+
+\subsubsection{User Interface}
+The interface was implemented using \emph{HTML} served using the View subsystem in Django.
+These views aim to create a usable system. These are based on the designs that were created
+during the \emph{PICTIVE} session. Slight alterations were made to the design to support required
+functionality that was not considered during the session. The interface comprises out of several
+views.
+\begin{description}
+\item[Task Overview] \hfill \\
+ This view has the purpose showing a user which tasks are outstanding. As well as show what is
+ happening within the team. This also provides links to the task control page for each task.
+ An additional feature that was implemented allows users to filter tasks based on a choice
+ of sites.
+\item[Task Control] \hfill \\
+ This is the view that non-privileged users will be interacting with most often. It provides
+ a detailed overview of what is required to be done with the task. Privileged users also
+ get a link to edit the task. This page also contains tools to allow users to download
+ and upload required files. Once a user is finished with the task at hand he/she can
+ simply press on the \emph{Finish Task} button. This triggers the transfer back routine
+ and queues the task for verification. Users who are able to verify the task will use the
+ same view however the \emph{Finish Task} button is replaced with two buttons that will
+ accept or reject the task. Logging information is also made visible to ensure that the
+ complete task history is visible.
+
+ This view is also used for server tasks. In this case the emphasis is on failure control.
+ With the log visible user can trace the source of the failure and rerun the task as soon
+ as it is has been dealt with.
+
+\item[Task Editing] \hfill \\
+ During the initial task setup the parameters of a task need to be set. This screen enables
+ the user to set the who the task is assigned to, what the input files are, and what the task
+ depends on. Note that the dependencies can also be set on in a friendly way on the task interface.
+ When something goes wrong with a tasks privileged users are also able to edit these setting
+ when the workflow is already active. The workflow should be able to dynamically adapt in this case.
+\item[Site View] \hfill \\
+ This view is dominated by the visualisation of the site. The visualisation was implemented using
+ \emph{jsPlumb}\footnote{jsPlumb Graph UI: jsPlumb.org}. This an API based that builds interactive
+ graphs. Each node provides and overview of the task. Dependencies are represented with edges
+ while the colour of the node indicates the status of task. Red indicates that there is a
+ problem with the task, grey indicates that the task has not been run, blue indicates the
+ task is in progress and green indicates that the task has been successfully completed. Dependencies
+ can be added by dragging an edge from corner to another task.
+
+ Below the visualisation is a list of the tasks in the site. This gives an overview
+ similar to the task overview screen. A privileged user can tasks off given jobs
+ from here which will then forward to the task editing screen.
+ As soon as the tasks in a site has been set up, the workflow can be started from this view.
+
+ The final set of components is there to add/edit jobs and categories in the system. These
+ are not specifically linked to a site and as such can be used throughout the system allowing
+ for easy reuse.
+\end{description}
+In addition to \emph{jsPlumb}, \emph{jQuery} and \emph{jQuery UI} is used extensively to build
+the client side interface.
+\subsection{Evaluation}
+
-\section{Iterataion 3\label{iteration3}}
+\section{Iteration 3\label{iteration3}}
\bibliography{bibliography}{}
Please sign in to comment.
Something went wrong with that request. Please try again.