Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/source/cheat_sheet.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*********************
FIREWHEEL Cheat Sheet
*********************

.. raw:: html

<p>You can download the cheat sheet here: <a href="_static/FIREWHEEL_Cheat_Sheet_final.pdf" target="_blank">FIREWHEEL Cheat Sheet</a></p>

70 changes: 56 additions & 14 deletions docs/source/cli/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,64 @@
Command Line Interface
######################

The FIREWHEEL Command Line Interface (CLI) allows interaction with and management of FIREWHEEL.
The CLI (:py:mod:`cli/firewheel_cli.py <firewheel.cli.firewheel_cli>`) uses Python's
:py:mod:`cmd` module.
However, there is one notable difference: the CLI is designed to work with a FIREWHEEL cluster which means that commands may need to be executed on one or many hosts.
To accomplish this, the CLI has been extended with :ref:`cli_helper_section`.
:ref:`cli_helper_section` do not use the standard `Cmd <https://docs.python.org/3/library/cmd.html>`_ format, but they enable us to use Python and Shell scripting to perform various actions across the cluster (see :ref:`cli_executors`).
The CLI automatically distributes Helpers to enable performing actions over the entire cluster.
Therefore, the CLI may be accessed from any node in the cluster.
The distinction between commands and Helpers is not relevant for most users so we will use these terms interchangeably.
CLI commands may output error message to the screen, but remote commands will indicate an error through a non-zero exit code.
Once the Python virtual environment is active, the CLI can be accessed by using the command ``firewheel`` in your terminal or directly using :exc:`firewheel_cli.py <firewheel.cli.firewheel_cli>`.
This will open up the CLI in "interactive" mode which looks like:

.. code-block:: bash

$ firewheel
FIREWHEEL Infrastructure CLI
fw-cli>

The CLI also supports single-command execution (or non-interactive mode).
This is the most common use of the CLI.
Running the CLI with arguments results in those arguments being treated as a single entered command (and associated arguments).
For example, the following examples are equivalent:

.. code-block:: bash

$ firewheel vm mix

.. code-block:: bash

fw-cli> vm mix

Helpers can be invoked by name as a command or using the :ref:`command_run` command.

.. code-block:: bash

fw-cli> run vm mix

While it is most common that users will invoke a Helper via its name, using the :ref:`command_run` command also disambiguates a Helper's name from built-in command names.
Additionally in interactive mode, the :ref:`command_run` command provides the ability to tab-complete Helpers.

Available Helpers can be enumerated using the :ref:`command_list` or :ref:`command_help` commands or they can found here: :ref:`cli_helpers`.

For complete help on any command, use: ``firewheel help <command | Helper>``.

Some commands may take arguments.
Everything on the command line after the name of the command is treated as an argument to that command.
Some commands use `Argparse <https://docs.python.org/3/library/argparse.html>`_ to handle arguments.
In this case, using ``-h`` or ``--help`` flag may also work.
However, this information *should* also be available by using: ``firewheel help <command | Helper>``.

All CLI commands *should* report success/failure via the exit code.
Many commands also output useful information directly to the screen.
For example:

.. code-block:: bash

$ firewheel repository list
Installed Model Component Repositories:
/opt/firewheel/model_components

As with other FIREWHEEL components, the CLI logs to ``firewheel.log``.
However, because the CLI is verbose, it has an additional log file (``cli.log``) which contains more in-depth logging information, including the output.
Additionally, the history of CLI commands is located in ``cli_history.log`` (or it can be accessed via the :ref:`command_history` command).

The location of the CLI-specific log files can be set using FIREWHEEL's configuration.

.. toctree::
:hidden:

usage.rst
cli_design.rst
cli_extention.rst
commands.rst
helper_docs.rst
61 changes: 0 additions & 61 deletions docs/source/cli/usage.rst

This file was deleted.

12 changes: 0 additions & 12 deletions docs/source/concepts.rst

This file was deleted.

6 changes: 3 additions & 3 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -340,9 +340,9 @@ def make_index(self, basepath):
# Get the list of model components
index = """.. _available_model_components:

#############################
Model Component Documentation
#############################
##########################
Available Model Components
##########################

"""

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
15 changes: 15 additions & 0 deletions docs/source/developer/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
##########
Developers
##########

.. toctree::
:maxdepth: 2

contributing
cli_extension
git-lfs
code_of_conduct
security
license
changelog

26 changes: 0 additions & 26 deletions docs/source/experiment/experiment.rst

This file was deleted.

28 changes: 19 additions & 9 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,33 @@ FIREWHEEL Documentation

.. toctree::
:hidden:
:caption: FIREWHEEL
:maxdepth: 2

emulytics
system/index
quickstart
concepts
install/index
system/index
tutorials/index
model_components/index
cli/index
code/index
developer/contributing
developer/code_of_conduct
developer/license
developer/security
developer/changelog

.. toctree::
:hidden:
:caption: Developers
:maxdepth: 2

model_components/index
developer/index
developer/code/index
glossary

.. toctree::
:hidden:
:caption: Resources
:maxdepth: 1

cheat_sheet

.. only:: html

******************
Expand Down
72 changes: 50 additions & 22 deletions docs/source/system/architecture.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
.. _FIREWHEEL-architecture:

Software Architecture
======================
*********************
Software Architecture
*********************

FIREWHEEL provides capabilities for the automated orchestration of VM deployments, virtual network provisioning, host/device application management, run time activity execution, and data collection for user defined cyber experiments.
To accomplish this, FIREWHEEL's *Control* system interprets user defined experiment models and interfaces with several open source technologies that handle things such as computer hardware virtualization, network virtualization, and persistent data storage, to name a few.
Expand All @@ -20,39 +21,62 @@ These FIREWHEEL software components are executed within a Python Virtual Environ
We'll next discuss each of FIREWHEEL's major software components.

.. note::

For more detailed insight into how FIREWHEEL's components interact with one another, see the full :ref:`class_diagrams` and :ref:`package_diagrams`.

.. _command-line-interface:
.. toctree::
:hidden:

diagrams

.. _fw_cli:

Command Line Interface
----------------------
======================

Users interact with FIREWHEEL's Command Line Interface (CLI) to task FIREWHEEL to execute its functions.
The CLI has a set of built in commands that users can use to do things like generate the CLI documentation and list available Helpers, as well as Helpers which extend the set of commands available through the CLI.
CLI commands and Helpers can be executed either directly from the console command line, or from within the CLI's own interactive shell environment. Since the CLI is designed to work with a FIREWHEEL cluster, its commands and Helpers will execute across cluster nodes to accomplish their tasks.
For more information about the commands and Helpers available via FIREWHEEL's CLI and how to use them, see the CLI documentation which is located in the CLI git repository.
The FIREWHEEL Command Line Interface (CLI) allows interaction with and management of FIREWHEEL.
The CLI (:py:mod:`cli/firewheel_cli.py <firewheel.cli.firewheel_cli>`) uses Python's
:py:mod:`cmd` module.
However, there is one notable difference: the CLI is designed to work with a FIREWHEEL cluster which means that commands may need to be executed on one or many hosts.
To accomplish this, the CLI has been extended with :ref:`cli_helper_section`.
:ref:`cli_helper_section` do not use the standard `Cmd <https://docs.python.org/3/library/cmd.html>`_ format, but they enable us to use Python and Shell scripting to perform various actions across the cluster (see :ref:`cli_executors`).
The CLI automatically distributes Helpers to enable performing actions over the entire cluster.
Therefore, the CLI may be accessed from any node in the cluster.
The distinction between commands and Helpers is not relevant for most users so we will use these terms interchangeably.
CLI commands may output error message to the screen, but remote commands will indicate an error through a non-zero exit code.

.. _experiment-models:

Experiment Models
-----------------
.. note::

For more information on the design of the FIREWHEEL CLI, please see :ref:`cli_design`.

Experiment models contain the details for all the elements to be included in a FIREWHEEL experiment.
This includes the definition of the experiment network topology, node and edge type definitions, VM assignments and specifications for each node in the topology and properties for every edge connection.
Experiment models can also contain VM resources, which are used to modify VMs after they've been launched, and the scheduling information that specifies where, when, and how each resource is to be used.
.. _experiment-models:

Model Components
^^^^^^^^^^^^^^^^
================
A FIREWHEEL experiment is simply a collection of model components which, when combined, define everything about an experiment.
The model components that make up an experiment define its network topology, which can include:

* Its vertices, which will become computing, networking, or other device VMs
* The edges that represent network connections amongst the vertices
* :py:class:`Vertex <firewheel.control.experiment_graph.Vertex>` and :py:class:`Edges <firewheel.control.experiment_graph.Edge>` class types that can be assigned to individual vertices
* The VM images associated with the various :py:class:`Vertex <firewheel.control.experiment_graph.Vertex>` class types
* VM resource files that will alter the state of a VM once booted
* Scheduling info for when and where those VM resources will modify the VM

FIREWHEEL's model components depend on one another to provide reusable, modular building blocks for constructing a wide variety of LAN, WAN, or even Internet scale experiments.

Model components are the building blocks of FIREWHEEL experiments.
Users build experiment models by creating and combining the sets of model components that define the topologies, attributes, configurations, and scheduled actions for their experiments.
Model components can depend on the outputs or capabilities of other model components, and ultimately the set of model components that make up a given experiment depend on model components provided by FIREWHEEL's Base Model Components repository.
For more information on Model Components, see :ref:`model_components`.
Model components can depend on the outputs or capabilities of other model components, and ultimately the set of model components that make up a given experiment depend on model components provided by `firewheel_repo_base <https://github.com/sandialabs/firewheel_repo_base>`_.

.. note::

For more information on the design of the Model Components, please see :ref:`mc_design`.

.. _control-system:

*Control*
---------
=========

Once an experiment has been defined through a set of model components, it's ready to be launched and instantiated as VMs and virtual networks.
The first step in this process is to translate the experiment's model, as described by the model components that it's comprised of, into an internal in-memory representation of the experiment's network topology.
Expand All @@ -77,7 +101,7 @@ FIREWHEEL uses `NetworkX <https://networkx.org/>`_ [#netx]_, a Python library fo
.. _minimega:

minimega
--------
========

`minimega <https://www.sandia.gov/minimega/>`__, is the name of the virtualization management component that's currently available with FIREWHEEL. minimega will instantiate an experiment using QEMU/KVM for VMs, and OVS networks.
minimega receives an experiment graph from FIREWHEEL's *Control* system, then determines and sends the appropriate set of instructions to the QEMU/KVM hypervisor and OVS virtual networking systems that are needed to instantiate the experiment model as an emulated computer network.
Expand All @@ -86,7 +110,7 @@ At this stage the experiment network's topology has been fully instantiated on t
.. _vm-resource-manager:

*VM Resource Manager*
---------------------
=====================

The last step in the process of launching a FIREWHEEL experiment is to monitor and manage the execution of any scheduled actions that need to be performed on the experiment network.
Scheduled actions can be separated into two temporal categories, pre and post experiment start.
Expand All @@ -95,8 +119,12 @@ The *VM Resource Manager* is the FIREWHEEL component that performs this job.
It receives the information about all actions that need to be performed (i.e. the vertices to perform them on, when they are to be performed, the commands, scripts or executables that need to be run, and/or any other resources required for accomplishing the action), and manages the execution of each action on each VM as required and at the designated time.
Once the *VM Resource Manager* has successfully finished monitoring and managing pre-start-time scheduled actions, then the experiment's emulated computer network is ready for conducting the intended experiment, and the *VM Resource Manager* will now do the same for actions that are scheduled to happen post-start-time i.e. actions that are part of the experiment.

.. note::

For more information on the design of the *VM Resource Manager*, please see :ref:`vm_resource_system`.

Analytics
---------
=========

Once the experiment is launched, gathering and analyzing experimental data becomes crucial.
To facilitate this process, FIREWHEEL provides seamless logging of VM resource output and generates JSON-formatted logs that can be easily ingested into data analysis tools such as Elasticsearch or Jupyter Notebooks.
Expand Down
File renamed without changes.
Loading
Loading