Skip to content

Commit

Permalink
Updates to scheduling docs
Browse files Browse the repository at this point in the history
  • Loading branch information
awicenec committed Nov 23, 2021
1 parent f7b7e45 commit 7b3be1a
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 4 deletions.
21 changes: 18 additions & 3 deletions daliuge-translator/dlg/dropmake/web/graph_init.js
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ function graphInit(type){
'fontSize': 10,
'offset': [-20,-20],
'fontWeight' : 400,
'color':'black',
'color': element.group.toString(),
// 'textBorderColor':'black',
// 'textBorderWidth' : 2.5,
// 'textBorderType' : 'solid'
Expand All @@ -92,7 +92,7 @@ function graphInit(type){
'fontSize': 10,
'fontWeight' : 900,
'fontStyle' : 'bold',
'color':'black',
'color': element.group,
"position":"inside",
// 'textBorderColor':'black',
// 'textBorderWidth' : 2.5,
Expand All @@ -107,10 +107,25 @@ function graphInit(type){
}
else {
newElement.name = element.key.toString();
newElement.color = 'black';
graphDataParts.nodeDataArray.push(newElement);
}
});

var numGroups = graphDataParts.nodeDataArray.length;
var spread = 255/Math.ceil((numGroups/3));
var ind = 0;
graphDataParts.nodeDataArray.forEach(element => {
var icol = Math.floor((256**(ind/(numGroups/3)) -1) * spread);
element.color = "#" + icol.toString(16).padStart(6, '0');
ind += 1;
})
graphData.nodeDataArray.forEach(element => {
var group = graphDataParts.nodeDataArray.filter(
function (item) {
return item.name == element.label.color
});
element.label.color = group[0].color
})
data.linkDataArray.forEach(element => {
newElement = {};
newElement.source = keyIndex.get(element.from);
Expand Down
2 changes: 1 addition & 1 deletion daliuge-translator/run_translator.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ case "$1" in
"dev")
export VCS_TAG=`git rev-parse --abbrev-ref HEAD| tr '[:upper:]' '[:lower:]'`
echo "Running Translator development version in foreground..."
docker run --volume $PWD/dlg/dropmake:/root/dlg/lib/python3.8/site-packages/dlg/dropmake --name daliuge-translator --rm -td -p 8084:8084 icrar/daliuge-translator:${VCS_TAG}
docker run --volume $PWD/dlg/dropmake:/root/dlg/lib/python3.8/site-packages/dlg/dropmake --name daliuge-translator --rm -t -p 8084:8084 icrar/daliuge-translator:${VCS_TAG}
exit 0;;
"casa")
export VCS_TAG=`git rev-parse --abbrev-ref HEAD| tr '[:upper:]' '[:lower:]'`-casa
Expand Down
2 changes: 2 additions & 0 deletions docs/architecture/graphs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ currently attempts to address the following optimisation goals:

* **Minimise the number of DataIslands** but subject to (1) a given **completion time deadline**, and (2) a given *DoP* (e.g. number of cores per node) that each DataIsland is allowed to take advantage of. In this problem, both completion time and resource footprint become the minimisation goals. The motivation of this problem is clear. In an scenario where two different schedules can complete the processing pipelinewithin, say, 5 minutes, the schedule that consumes less resources is preferred. Since a DataIsland is mapped onto resources, and its capacity is already constrained by a given DoP, the number of DataIslands is proportional to the amount of resources needed. Consequently, schedules that require less number of DataIslands are superior. Inspired by the `hardware/software co-design <http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=558708>`_ method in embedded systems design, |daliuge| uses a "look-ahead" strategy at each optimisation step to adaptively choose from two conflicting objective functions (deadline or resource) for local optimisation, which is more likely to lead to the global optimum than greedy strategies.

In addition to the automatic deployment and scheduling options, there is also a special construct component available, called 'Exclusive Force Node', to allow users to enforce the placement of certain parts of the graph on a single compute node (NOTE: This is still work-in-progress.). In the case that a scattered section of the graph is enclosed in such an Exclusive Force Node construct, each of the scattered sections will be deployed on a compute node. In case there are not enough compute nodes available to accommodate all the scattered sections, some of them might be deployed (in whole, but together) on a single node. This also shows the risk of using such 'hints': It essentially reduces the degrees of freedom of the scheduling algorithm(s) and thus might turn out to be less optimal at runtime.

Physical Graph
^^^^^^^^^^^^^^

Expand Down

0 comments on commit 7b3be1a

Please sign in to comment.