Skip to content

Commit

Permalink
docs: add developer documentation for reservation
Browse files Browse the repository at this point in the history 
This also switches PlantUML's output format to
"svg_img", so the large activity diagrams are not
truncated.
  • Loading branch information
@arogge
arogge committed Mar 6, 2019
1 parent 28e8485 commit 9136a05
Show file tree
Hide file tree
Showing 6 changed files with 407 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/manuals/en/new_main_reference/source/conf.py
View file
Expand Up @@ -245,3 +245,5 @@

#scv_root_ref = 'dev/pstorz/bareos-18.2/sphinx-versioning'
#scv_banner_main_ref = 'dev/pstorz/bareos-18.2/sphinx-versioning'

plantuml_output_format = 'svg_img'
1 change: 1 addition & 0 deletions docs/manuals/en/new_main_reference/source/developers.rst
View file
Expand Up @@ -21,3 +21,4 @@ Bareos Developer Guide
developers/netprotocol.rst
developers/directorConsole.rst
developers/smartall.rst
developers/reservation.rst
35 changes: 35 additions & 0 deletions docs/manuals/en/new_main_reference/source/developers/reservation.rst
View file
@@ -0,0 +1,35 @@
Reservation
===========

Introduction
------------
After the director set up a job on the SD it will then issue a "use" command to tell the SD what storage device it wants to use.
The SD then checks the list of devices the director sent and tries to reserve a suitable device.
In the end the SD either tells the director which device to use or that there is no device available for the job.

There is a so-called reservation context that is passed to every function that has a swimlane in this diagram.
This context is basically a bunch of flags that are set and cleared to change the behaviour of the rest of the code.

For example CanReserveDevice() from the second diagram sets the flag low_use_drive which is then evaluated in UseDeviceCmd().

Diagram Conventions
~~~~~~~~~~~~~~~~~~~
Each swimlane denotes a separate function.

.. include:: reservation/legend.puml

UseDeviceCmd
~~~~~~~~~~~~
This function reads the list of storages and devices the director is willing to use for this job.
Afterwards several different methods of finding a device to reseve are used.
If no device could be reserved the function waits for up to a minute or until ReleaseDeviceCond() is called and then tries again.

.. include:: reservation/UseDeviceCmd.puml

ReserveDevice
~~~~~~~~~~~~~
Here we see wether the media type matches and actually try to open the device.
The actual reservation is delegated to ReserveDeviceForRead() or ReserveDeviceForAppend().
While the first one is more or less trivial, the latter one is really complicated.

.. include:: reservation/ReserveDevice.puml
154 changes: 154 additions & 0 deletions docs/manuals/en/new_main_reference/source/developers/reservation/ReserveDevice.puml
View file
@@ -0,0 +1,154 @@
.. uml::
:caption: Control flow of ReserveDevice()

@startuml
|ReserveDevice|
start
#limegreen:if (device matches requested media_type?) then (yes)
if (device initialized?) then (no)
:InitDev()|
endif
if (device exists and is open?) then (yes)
#aqua:set suitable_device]
:initialize DCR;
if (append?) then (no)
:ReserveDeviceForRead()|
else (yes)
|ReserveDeviceForAppend|
if (CanRead()) then (no)
if (IsDeviceUnmounted()) then (no)
|CanReserveDrive|
if (IsMaxJobsOk()) then (yes)
#aqua:if (any_drive set?) then (no)
#aqua:if (try_low_use_drive set?) then (yes)
#aqua:if (low_use_drive is current drive?) then (yes)
(S)
detach
endif
endif
#aqua:if (PreferMountedVols set?) then (no)
if (isBusy()) then (yes)
#aqua:if (dev.num_writers + NumReserved() < num_writers) then (yes)
#aqua:set num_writers := dev.numwriters + NumReserved()]
#aqua:set low_use_drive := dev]
endif
(X)
detach
endif
else (yes)
if (IsTape() but no volume) then (yes)
(X)
detach
endif
endif
#aqua:if(exact_match set? and have_volume set?) then (yes)
#aqua:if(VolumeName matches mounted volume) then (no)
(X)
detach
endif
if (Can_i_use_volume()) then (no)
(X)
detach
endif
endif
:line 1080;

:line 1152;
endif
#aqua:if (autochanger_only) then (yes)
if (IsBusy()) then (no)
if (volume in drive) then (no)
(S)
detach
endif
endif
endif
if (num_writers == 0) then (yes)
if (NumReserved()) then (yes)
if (IsPoolOk()) then (yes)
(S)
detach
else (no)
(X)
detach
endif
elseif (CanAppend()) then (yes)
if (IsPoolOk()) then (no)
:UnloadAutoChanger()|
endif
(S)
detach
endif
endif
if (num_writers > 0 || CanAppend()) then (yes)
if (IsPoolOk()) then (yes)
(S)
#tomato:return success;
else (no)
(X)
detach
endif
else (no)
#tomato:cancel job (M_FATAL);
detach
endif
else (no)
(X)
#tomato:return failure;
endif
|ReserveDeviceForAppend|
if (success?) then (yes)
:reserve device;
:set have_device]
endif
endif
endif
|ReserveDevice|
if (have_device?) then (yes)
#aqua:if (have_volume set?) then (yes)
if (reserve_volume()) then (no)
(F)
detach
endif
else (no)
if (DirFindNextAppendableVolume()) then (yes)
#aqua:set have_volume]
#aqua:set VolumeName]
else (no)
#aqua:clear have_volume]
#aqua:clear VolumeName]
if (FoundInUse()) then (yes)
#aqua:if(PreferMountedVols set?) then (no)
#aqua:set PreferMountedVols]
if (dcr has volume?) then (yes)
:UnreserveDevice()|
endif
(F)
detach
endif
endif
if (num_writers != 0) then (yes)
if (dcr has volume?) then (yes)
:UnreserveDevice()|
endif
(F)
detach
endif
endif
endif
else (no)
(F)
detach
endif
endif
if (have_device?) then (yes)
:OK device message>
else (no)
(F)
#aqua:clear have_volume]
#aqua:clear VolumeName]
endif
endif
endif

@enduml
191 changes: 191 additions & 0 deletions docs/manuals/en/new_main_reference/source/developers/reservation/UseDeviceCmd.puml
View file
@@ -0,0 +1,191 @@
.. uml::
:caption: Control flow of UseDeviceCmd()

@startuml
'BEGIN macros
!definelong RESERVE_DEVICE()
:ReserveDevice()|
!enddefinelong

!definelong AUTOCHANGER()
partition autochanger {
while (with each device in autochanger device)
#limegreen:if (device autoselect?) then (yes)
RESERVE_DEVICE()
if (have_device?) then (yes)
#tomato:return;
detach
endif
endif
endwhile
}
!enddefinelong

!definelong PLAIN_DEVICE()
partition plain_device {
RESERVE_DEVICE()
note right
try to reserve the named device
end note
if (have_device?) then (yes)
#tomato:return;
detach
endif
#limegreen:if (DeviceReserveByMediatype) then(yes)
while (each device with matching media_type)
RESERVE_DEVICE()
if (have_device?) then (yes)
#tomato:return;
detach
endif
endwhile
endif
}
!enddefinelong

!definelong MOUNTED_VOLUMES()
partition mounted_volumes {
if (append?) then(yes)
while(with each mounted volume)
:ask director if volume is ok/
if (volume ok?) then(yes)
:find device for volume;
RESERVE_DEVICE()
if (have_device?) then (yes)
#tomato:return;
detach
endif
endif
endwhile
endif
}
!enddefinelong
'END macros
|UseDeviceCmd|
start
partition read_data_from_director {
repeat
:storage specification<
repeat
:device specification<
repeat while (more devices?)
repeat while (more storages?)
}

repeat
#aqua:clear suitable_device]
#aqua:clear have_volume]
#aqua:clear VolumeName]
#aqua:clear any_drive]
#limegreen:if (PreferMountedVols) then (no)
#aqua:clear low_use_drive]
#aqua:clear PreferMountedVols]
#aqua:clear exact_match]
#aqua:set autochanger_only]
|FindSuitableDeviceForJob|
while (with every device)
|SearchResForDevice|
AUTOCHANGER()
|FindSuitableDeviceForJob|
endwhile
|UseDeviceCmd|
if (have_device) then (yes)
stop
else (no)
#aqua:if (low_use_drive set?) then (yes)
#aqua:set try_low_use_drive]
|FindSuitableDeviceForJob|
while (with every device)
|SearchResForDevice|
AUTOCHANGER()
|FindSuitableDeviceForJob|
endwhile
|UseDeviceCmd|
if (have_device) then (yes)
stop
endif
#aqua:clear try_low_use_drive]
endif
#aqua:clear autochanger_only]
|FindSuitableDeviceForJob|
while (with every device)
|SearchResForDevice|
AUTOCHANGER()
PLAIN_DEVICE()
|FindSuitableDeviceForJob|
endwhile
|UseDeviceCmd|
if (have_device) then (yes)
stop
endif
endif
else (yes)
endif
#aqua:set PreferMountedVols]
#aqua:set exact_match]
#aqua:clear autochanger_only]
|FindSuitableDeviceForJob|
MOUNTED_VOLUMES()
while (with every device)
|SearchResForDevice|
AUTOCHANGER()
PLAIN_DEVICE()
|FindSuitableDeviceForJob|
endwhile
|UseDeviceCmd|
if (have_device) then (yes)
stop
else (no)
#aqua:clear exact_match]
|FindSuitableDeviceForJob|
MOUNTED_VOLUMES()
while (with every device)
|SearchResForDevice|
AUTOCHANGER()
PLAIN_DEVICE()
|FindSuitableDeviceForJob|
endwhile
|UseDeviceCmd|
if (have_device) then (yes)
stop
else (no)
#aqua:set any_drive]
|FindSuitableDeviceForJob|
MOUNTED_VOLUMES()
while (with every device)
|SearchResForDevice|
AUTOCHANGER()
PLAIN_DEVICE()
|FindSuitableDeviceForJob|
endwhile
|UseDeviceCmd|
if (have_device) then (yes)
stop
else (no)
if (attempt 3+?) then (yes)
:wait 30 seconds;
else (no)
#aqua:if (suitable_device set?) then (yes)
:WaitForDevice()|
note right
This will acquire a mutex to queue up
multiple jobs waiting for a device.
Then it waits up to 60 seconds for some
other thread to call ReleaseDeviceCond()
end note
else (no)
(F)
detach
endif
endif
endif
endif
endif
repeat while (repeat forever)
detach
partition failed_to_reserve {
(F)
:no device message>
stop
}
@enduml

0 comments on commit 9136a05

Please sign in to comment.