Docs for the director-envoy 1.2 update #154
Conversation
|
ok to test |
igor-davidyuk
force-pushed
the
docs-director-envoy
branch
from
86db16c
to
5148968
Compare
docs/source/openfl/components.rst
Outdated
| ========= | ||
|
|
||
| |productName| comes with another long-existing actor called Envoy. It runs on collaborator machines connected to a *Director*. | ||
| There is one to one mapping between *Envoys* and Dataset shards: every *Envoy* needs exactly one *Shard Descriptor* to run. |
There was a problem hiding this comment.
This makes the ontology between the Envoy and Datasets very clear. It might also be helpful to provide a link to the ShardDescriptor here
There was a problem hiding this comment.
I believe the place, where ShardDescriptor is explained, is director-based workflow, I can place a link to that section.
Alternatively, we could create another section under OpenFL structure that would describe the interface entities for collaborator nodes owners / managers
There was a problem hiding this comment.
For now I added a link to the ShardDescriptor base class
docs/source/openfl/components.rst
Outdated
| ============= | ||
|
|
||
| Collaborator is also a short living entity, it manages training the model on local data: executes assigned tasks, converts DL framework-specific tensor objects to |productName| inner representation, and exchanges model parameters with the aggregator. | ||
| Converting tensors is done by Framework adapter plugins. |productName| ships with Pytorch and TensorFlow 2 framework adapters, this list will be extended in the future. User is free to implement their adapter for the required DL framework enabling |productName| support for experiments using this framework. The adapter plugin interface is simple: there are two required methods to load and extract tensors from a model and optimizer. Model is loaded with relevant weights before every task and at the end of the training task, weights are extracted to be sent to the central node and aggregated. |
There was a problem hiding this comment.
"|productName| ships with Pytorch and Tensorflow 2.x framework adapters. These framework adapters are intended to be extensible, and we encourage users to contribute new adapters for DL frameworks they would like to see supported in |productName|."
There was a problem hiding this comment.
Is the Framework Adapter documented elsewhere? The content is good here, but we should add a dedicated section to the documentation that walks through the plugin interface
There was a problem hiding this comment.
Accepted proposed changes and added a dedicated section for plugin components.
https://openfl.readthedocs.io/en/docs-director-envoy/source/openfl/plugins.html
|
Please merge the latest changes from develop branch in order to pass new CI tests. |
also restructured PKI and the old workflow
igor-davidyuk
force-pushed
the
docs-director-envoy
branch
from
1f0c4bb
to
f02a9de
Compare
|
|
||
| |productName| comes with another long-existing actor called *Envoy*. It runs on collaborator machines connected to a *Director*. | ||
| There is one to one mapping between *Envoys* and Dataset shards: every *Envoy* needs exactly one | ||
| `Shard Descriptor <https://github.com/intel/openfl/blob/develop/openfl/interface/interactive_api/shard_descriptor.py>`_ to run. |
docs/source/openfl/components.rst
Outdated
| =========== | ||
|
|
||
| The *Aggregator* is a short-living entity, which means that its lifespan is limited by experiment execution time. | ||
| It orchestrates *Collaborators* according to the FL plan and performs model updates aggregation. |
There was a problem hiding this comment.
"...performs model aggregation at the end of each round".
docs/source/openfl/plugins.rst
Outdated
| @staticmethod | ||
| def set_tensor_dict(model, tensor_dict, optimizer=None, device='cpu') -> None: | ||
|
|
||
| Implement :code:`serialization_setup` optional method if some preparation is required before the model serialization. |
There was a problem hiding this comment.
"If your new framework model cannot be directly serialized with pickle-type libraries, you can optionally implement the :code:serialization_setup method to prepare the model object for serialization."
There was a problem hiding this comment.
replaced, thank you
docs/source/utilities/pki.rst
Outdated
|
|
||
| If you have trusted workspace and connection should not be encrypted you can use :code:`disable_tls` option while starting experiment. | ||
| Otherwise it is necessary to certify each node participating in the federation. Certificates allow to use mutual tls connection between nodes. | ||
| You can certify nodes by your own pki system or use pki provided by OpenFL. It is based on `step-ca <https://github.com/smallstep/certificates>`_ |
There was a problem hiding this comment.
PKI should be capitalized everywhere except shell commands (i.e. fx pki ...)
No description provided.