add envars + packaging docs, and hierarchy changes in doc

docs/source/best-practices.md

@@ -151,3 +151,11 @@ In case of [StopActorExecutionError](pydoc/leapp.html#leapp.exceptions.StopActor
 - [ReportOnly](pydoc/leapp.workflows.html?highlight=FailPhase#leapp.workflows.policies.Policies.Errors.ReportOnly) - do not end the workflow execution at all and continue with logging the issue only
 
 You can also use the [StopActorExecution](pydoc/leapp.html#leapp.exceptions.StopActorExecution) and [StopActorExecutionError](pydoc/leapp.html#leapp.exceptions.StopActorExecutionError) exceptions inside a private or shared library.
+
+## Use the LEAPP and LEAPP\_DEVEL prefixes for new envars
+
+In case you need to change a behaviour of actor(s) for testing or development purposes - e.g. be able to skip a functionality in your actor - use environment variables. Such environment variables should start with prefix *LEAPP\_DEVEL*. Such variables are not possible to use on production systems without special *LEAPP\_UNSUPPORTED* variable. This prevents users to break their systems by a mistake.
+
+If you want to enable user to modify behaviour on production systems as well (e.g. enable user increase size of something the actor is producing), use just *LEAPP* prefix and ensure you inform user about the variable if needed.
+
+In both cases, such environment variables should be mentioned in related commit messages.

docs/source/compatibility-with-leapp-repository.md

@@ -0,0 +1,59 @@
+# Compatibility between leapp and leapp repositories
+
+There is a hidden dragon in the split of leapp framework and leapp repositories
+projects. The development of features and incompatible changes has different
+speed in both projects. We want to release our projects using semantic
+versioning and, at the same time, we do not want to release a new version of
+the project after each merge of a new feature or incompatible change.
+We rather prefer releasing new versions with changelogs and everything
+when we agree it's worthwhile.
+
+But we need a mechanism to be able to synchronize with other projects, when we
+provide new functionality in the upstream (master) branch without the need
+of immadiate release of the new version of leapp. For these purposes the
+`leapp-framework` capability is provided in the framework (python[23]-leapp) rpms.
+
+## When and how change the capability
+
+The `leapp-framework` capability has to be changed in case of any change of
+provided API (e.g. change in the leapp standard library) or functionality that
+could affect actors in leapp repositories. That includes new features as well
+as an actor may depend on that in future, and the leapp-repository package
+should be able to specify correct framework version that is needed.
+
+The `leapp-framework` capability uses just `X.Y` versioning:
+
+- In case of a change breaking previous functionality (e.g. changed a name of a function) provided by the framework, bump `X` and set `Y` to 0.
+- In case of a new feature (e.g. new function in stdlib), which doesn't affect already existing actors, bump `Y`
+
+The value of the capability should be bumped per PR providing all changes.
+IOW, when a PR implements two compatible/incompatible changes in several
+commits, the value should be bumped only once!
+
+As well it is recommended to mentioned the value of the capability
+(when changed) in the commit msg, so it the tracking is easy.
+
+## Suggested dependency in leapp-repository rpms
+We suggest to set dependency on the `leapp-framework` capability in any rpm
+that contains leapp repositories in this way:
+
+```spec
+Requires:       leapp-framework >= 1.2, leapp-framework < 2
+```
+
+Which means, that you want to install framework of version at least 1.2 but
+lower than 2 (e.g. 1.99).
+
+### Possible issue
+There is unfortunately one issue with the above solution. As far as there is just
+one rpm providing that capability, it's ok. The problem occurs on systems where
+there will be more rpms (of different name) providing the capability.
+So in case you are able to install e.g. python2-leapp and python3-leapp rpms
+on the system, you could end up with:
+
+- python2-leapp providing leapp-framework 1.0, and
+- python3-leapp providing leapp-framework 3.0
+
+which both are broken for leapp repository and the dependency from the point of
+rpms is satisfied. This should happen rarely. We suggest you to ensure that you
+use such repositories where only one of those rpms exists.

docs/source/actor-rhel7-to-rhel8.md → docs/source/el7toel8/actor-rhel7-to-rhel8.md

@@ -56,13 +56,13 @@ New actor MyNewActor has been created at /usr/share/leapp-repository/repositorie
 
 The main file of the actor is the actor.py in which you’ll write code for logic of the actor.
 
-For further information about how create an actor read this [document](first-actor).
+For further information about how create an actor read this [document](../first-actor).
 
 ## Including an actor in the RHEL 7 to 8 upgrade process
 
 Until now, you have created boilerplate of a new actor and made it visible to Leapp. But, Leapp needs some more information about what to do with the actor. Specifically, in which **“workflow”** and in which **“phase”** the actor should be executed. A workflow is a sequence of phases. The only workflow available now is the one solving the upgrade of RHEL 7 to RHEL 8. Each phase is a set of actors that will be executed one after another before the next phase starts. To find out in which workflow and phase should the actor be executed, Leapp looks for **“tags”**. To be part of RHEL 7 to RHEL 8 upgrade workflow, an actor needs to be tagged with **IPUWorkflowTag**.
 
-The phases of the IPUWorkflow (in order) are: **Facts Collection, Checks, Report, Download, Upgrade RamDisk Preparation, Upgrade RamDisk Start, Late Tests, Preparation, RPM Upgrade, Application Upgrade, Third Party Applications, Finalization** and **First Boot**. Each phase has a specific tag that marks an actor as being part of that phase. You can find descriptions of all the phases and their tags [here](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/workflows/inplace_upgrade.py) and workflow diagram [here](inplace-upgrade-workflow).
+The phases of the IPUWorkflow (in order) are: **Facts Collection, Checks, Report, Download, Upgrade RamDisk Preparation, Upgrade RamDisk Start, Late Tests, Preparation, RPM Upgrade, Application Upgrade, Third Party Applications, Finalization** and **First Boot**. Each phase has a specific tag that marks an actor as being part of that phase. You can find descriptions of all the phases and their tags [here](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/workflows/inplace_upgrade.py) and workflow diagram [here](../inplace-upgrade-workflow).
 
 For example, if an actor is to be executed within the Checks phase, it needs to be tagged both with IPUWorkflowTag and ChecksPhaseTag. The result after updating the boilerplate would be:
 
@@ -88,7 +88,7 @@ class MyNewActor(Actor):
 
 All communication between actors in Leapp is carried out using **“messages”**. An actor can consume or produce messages. A message may contain any data, but the data needs to be in a specific format defined by a **“model”**. If an actor wants to consume a message produced by another actor, it needs to specify the specific model of the consumed messages. Leapp will make sure to execute such an actor only after some message of the specified model was produced by another actor. If no message of the specified model was produced in previous phases or in the current phase, the consuming actor will get no messages of that kind.
 
-For further information about messaging refer to this [document](messaging).
+For further information about messaging refer to this [document](../messaging).
 
 One of the existing models in Leapp is [ActiveKernelModulesFacts](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/models/activekernelmodulesfacts.py). Messages from this model contain data about the system on which Leapp has been started. For example, it contains installed kernel modules. If an actor wants to perform some action based on existing kernel modules on the system, the actor can get list of these modules by consuming the _ActiveKernelModulesFacts_ messages. By extending the boilerplate, the code could look like this:
 
@@ -132,7 +132,7 @@ The following restrictions apply:
   change the behavior of any other dialog that any actor might create.
 - Dialogs can be used only at certain stages of the workflow - ChecksPhase and TargetTransactionChecksPhase.
 
-For more information and real examples please check [dialogs](dialogs).
+For more information and real examples please check [dialogs](../dialogs).
 
 ### Producing data for other actors and reporting
 
@@ -359,11 +359,11 @@ To flush all saved messages from the repository database, run `snactor messages
 
 ## Writing tests for an actor
 
-[Read the tutorial](unit-testing) for writing an running unit and component tests
+[Read the tutorial](../unit-testing) for writing and running unit and component tests
 
 ## Best practices
 
-Read the best practices [document](best-practices) and [Python guidelines](https://github.com/oamg/leapp-guidelines/blob/master/python-coding-guidelines.md).
+Read the best practices [document](../best-practices) and [Python guidelines](https://github.com/oamg/leapp-guidelines/blob/master/python-coding-guidelines.md).
 
 ## Contributing actors to the Leapp project
 
@@ -373,15 +373,15 @@ All new content that needs to be part of Leapp release distributed to all users
 
 **Before submitting your work for review, make sure you have read and followed the contribution guidelines:**
 
-[Contribution guidelines for writing actors](contributing)
+[Contribution guidelines for writing actors](../contributing)
 
 This [pull request](https://github.com/oamg/leapp-repository/pull/186) gives a good example of both guidelines-driven actor implementation and thorough test coverage.
 
 ## FAQ
 
 ### In which existing workflow phase should I place my new actor?
 
-You can decide that based on the description of the phases this information is available in the [code](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/workflows/inplace_upgrade.py) and diagram [here](inplace-upgrade-workflow). Please note that if your actor depends on some message generated by another actor, it cannot be executed in a phase before the phase of such actor. In a similar way, if your actor produces data, it needs to be executed before the actor consuming the data.
+You can decide that based on the description of the phases this information is available in the [code](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/el7toel8/workflows/inplace_upgrade.py) and diagram [here](../inplace-upgrade-workflow). Please note that if your actor depends on some message generated by another actor, it cannot be executed in a phase before the phase of such actor. In a similar way, if your actor produces data, it needs to be executed before the actor consuming the data.
 
 ### How to stop the upgrade in case my actor finds a problem with the system setup?
 
@@ -414,7 +414,7 @@ Python 2.7+/3.6+, but keep in mind that the resulting code has to be both py2 an
 
 Under “tests” folder, an actor can have Python files containing tests that will be executed using PyTest. Leapp provide tests utilities to simulate actor consuming and checking actor production. Please, refer to detailed Leapp documentation about how to write tests.
 
-For further information read: [Writing tests for actors](unit-testing).
+For further information read: [Writing tests for actors](../unit-testing).
 
 ### How to use libraries or data files in my new actor?
 

docs/source/el7toel8/envars.md

@@ -0,0 +1,65 @@
+# Environment variables for el7toel8 repository
+
+Actors in the el7toel8 repository use environment variables specified below.
+All these envars use the suggested prefixes specified in
+[the best practices document](../best-practices.html#use-the-leapp-and-leapp-devel-prefixes-for-new-envars)
+for the leapp project to distinguish their purpose: *production* or *devel* use.
+
+If the argument for envars below is not specified, it is expected to set `0`
+(false) or `1` (true).
+
+## LEAPP_UNSUPPORTED
+
+Necessary to use in case you use any envar with the LEAPP_DEVEL prefix
+(see the list below). And in case you use the --whitelist-experimental option
+for the Leapp tool.
+
+
+## LEAPP_DEVEL_RPMS_ALL_SIGNED
+
+Leapp will consider all installed pkgs to be signed by RH - that affects
+the upgrade process as by default Leapp upgrades only pkgs signed by RH.
+Leapp takes care of the RPM transaction (and behaviour of applications)
+related to only pkgs signed by Red Hat. What happens with the non-RH signed
+RPMs is undefined.
+
+
+## LEAPP_DEVEL_TARGET_RELEASE
+
+Change the default target RHEL 8 minor version.
+
+
+## LEAPP_DEVEL_SKIP_RHSM
+
+Do not use `subscription-manager`. Using this option (now) requires creating
+a custom actor providing RHEL 8 repos through CustomTargetRepository message.
+
+
+## LEAPP_DEVEL_SKIP_CHECK_OS_RELEASE
+
+Do not check whether the source RHEL 7 version is the supported one.
+E.g. right now Leapp does not allow you to proceed with the upgrade
+when you’re not on RHEL 7.6.
+
+
+## LEAPP_DEVEL_DM_DISABLE_UDEV
+
+Setting the environment variable provides a more convenient
+way of disabling udev support in libdevmapper, dmsetup and LVM2 tools globally
+without a need to modify any existing configuration settings.
+This is mostly useful if the system environment does not use udev.
+
+
+## LEAPP_DEVEL_SOURCE_PRODUCT_TYPE
+
+By default the upgrade is processed from the GA (general availability) system
+using GA repositories. In case you need to do the in-place upgrade from
+Beta or HTB system, use the variable to tell which of those you would like
+to use. It's case insesitive. The `ga` value is the default.
+Expected values: `ga`, `beta`, `htb`.
+
+
+## LEAPP_DEVEL_TARGET_PRODUCT_TYPE
+
+Analogy to the LEAPP_DEVEL_SOURCE_PRODUCT_TYPE envar, just for the target
+system this time. Again, `ga` is the default.

docs/source/el7toel8/leapp-repository-el7toel8.rst

@@ -0,0 +1,10 @@
+Leapp repository for RHEL 7 to RHEL 8 upgrade
+=============================================
+This is the official upstream documentation for the leapp repository for in-place upgrade (IPU) from RHEL 7 to RHEL 8. The homepage of the project is `here <https://github.com/oamg/leapp-repository>`_.
+
+.. toctree::
+    :caption: Contents:
+
+    actor-rhel7-to-rhel8
+    inhibit-rhel7-to-rhel8
+    envars

docs/source/faq.md

@@ -23,7 +23,7 @@ Leapp project aims to enable users to modernize their existing workloads without
 
 ## How can I get on board with contributing to Leapp?
 
-For the Leapp framework we are currently developing the functionality for the in-place upgrade of RHEL 7 to RHEL 8. You can improve the user experience of the upgrade by creating so called actors for the Leapp framework. We've written a quick guide on how to create such actors for the RHEL 7 to RHEL 8 upgrades: [How to create a Leapp actor for RHEL 7 to 8 upgrade.](actor-rhel7-to-rhel8)
+For the Leapp framework we are currently developing the functionality for the in-place upgrade of RHEL 7 to RHEL 8. You can improve the user experience of the upgrade by creating so called actors for the Leapp framework. We've written a quick guide on how to create such actors for the RHEL 7 to RHEL 8 upgrades: [How to create a Leapp actor for RHEL 7 to 8 upgrade.](el7toel8/actor-rhel7-to-rhel8)
 
 ## What is an actor and what does it do?
 
@@ -38,13 +38,13 @@ In regards to the upgrades of RHEL 7 to RHEL 8, Leapp should be able to upgrade
 ## How can I exchange any data between actors?
 
 All communication between actors in Leapp is carried out using " messages". An actor can consume or produce messages. A message may contain any data, but the data needs to be in a specific format defined by a "model". If an actor wants to consume a message produced by another actor, it needs to specify the specific model of the consumed messages. Leapp will make sure to execute such an actor only after some message of the specified model was produced by another actor. If no message of the specified model was produced in previous phases or in the current phase, the consuming actor will get no messages of that kind.
-Source: [How to create a Leapp actor for RHEL 7 to 8 upgrade](actor-rhel7-to-rhel8)
+Source: [How to create a Leapp actor for RHEL 7 to 8 upgrade](el7toel8/actor-rhel7-to-rhel8)
 
 ## What do I have to do in order to execute actor I just wrote?
 
 If you want to execute just a single actor when developing it, then use the snactor tool. [Here's a tutorial](first-actor) on how to use it.
 If you want to add your actor to an existing workflow, for example the RHEL 7 to 8 upgrade workflow, then tag your actor with appropriate workflow and phase tags.
-Source: [How to create a Leapp actor for RHEL 7 to 8 upgrade](actor-rhel7-to-rhel8)
+Source: [How to create a Leapp actor for RHEL 7 to 8 upgrade](el7toel8/actor-rhel7-to-rhel8)
 
 ## What should I do if I need to execute multiple actors? Can I somehow ensure the dependencies between them?
 
@@ -78,7 +78,7 @@ It should follow the [Contribution guidelines](contributing) and the [Best pract
 ## How can I debug my actor? Is there a standard/supported way how to log and get logs from actors/channels?
 
 You can run your actor using the snactor tool and printing the output. [See the tutorial](first-actor) on how to use snactor.
-Source: [How to create a Leapp actor for RHEL 7 to 8 upgrade](actor-rhel7-to-rhel8)
+Source: [How to create a Leapp actor for RHEL 7 to 8 upgrade](el7toel8/actor-rhel7-to-rhel8)
 
 ## Are there some technical limitations for an actor? E.g. maximum time execution, size of the input/output, libraries I can use... In case there are, is it possible to specify that the actor needs e.g. longer time for execution?
 

docs/source/index.rst

@@ -12,12 +12,11 @@ Resources not included in this RTD:
 
     terminology
     tutorials
-    repository-dir-layout
+    leapp-repositories
+    infrastructure
     best-practices
     test-actors
     architecture
-    dependencies
-    dependencies-leapp-repository
     inplace-upgrade-workflow
     contributing
     python-coding-guidelines

docs/source/infrastructure.rst

@@ -0,0 +1,11 @@
+Infrastructure
+==============
+Here you can find documentation related to our CI/CD, packaging, etc.
+
+
+.. toctree::
+    :caption: Contents:
+
+    dependencies
+    dependencies-leapp-repository
+    compatibility-with-leapp-repository

docs/source/leapp-repositories.rst

@@ -0,0 +1,14 @@
+Leapp repositories
+==================
+
+Here you can find all the information related to
+`leapp repositories <terminology.html#repository>`_, including the documentation
+for existing leapp repositories managed by the OS and Application
+Modernization Group (`OAMG <https://github.com/oamg>`_)
+
+
+.. toctree::
+    :caption: Contents:
+
+    repository-dir-layout
+    el7toel8/leapp-repository-el7toel8

docs/source/tutorials.rst

@@ -7,8 +7,8 @@ Tutorials
     devenv-install
     create-repository
     first-actor
-    actor-rhel7-to-rhel8
-    inhibit-rhel7-to-rhel8
+    el7toel8/actor-rhel7-to-rhel8
+    el7toel8/inhibit-rhel7-to-rhel8
     messaging
     dialogs
     repo-linking