Skip to content
This repository has been archived by the owner on Dec 7, 2022. It is now read-only.
/ pulp Public archive

Commit

Permalink
Add docs on lazy to the workflows section
Browse files Browse the repository at this point in the history
  • Loading branch information
Brian Bouterse committed Dec 17, 2018
1 parent a3aaa5e commit 228ae96
Show file tree
Hide file tree
Showing 4 changed files with 79 additions and 12 deletions.
24 changes: 15 additions & 9 deletions docs/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ Glossary
A file that belongs to a :term:`content unit<content>`.

:class:`~pulpcore.app.models.Content`
The smallest unit of data that can be added and removed from :term:`repositories<repository>`.
When singular, "content unit" should be used. Content units can have multiple
:term:`artifacts<artifact>`. Each content unit has a :term:`type` (like .rpm or .deb) which
that is defined by a :term:`plugin`
The smallest unit of data that can be added and removed from
:term:`repositories<repository>`. When singular, "content unit" should be used. Content
units can have multiple :term:`artifacts<artifact>`. Each content unit has a :term:`type`
(like .rpm or .deb) which that is defined by a :term:`plugin`.

content app
A `Django <https://docs.djangoproject.com>`_ app provided by :term:`pulpcore` that serves
Expand All @@ -20,18 +20,23 @@ Glossary
User facing settings that specify how and where associated
:term:`publications<publication>` are served.

lazy content
A :term:`content unit<content>` that was synchronized into Pulp but is missing one or more
:term:`Artifacts<artifact>`. Lazy content is associated with a :term:`Remote` that knows how
to download those :term:`Artifacts<artifact>`.

plugin
A `Django <https://docs.djangoproject.com>`_ app that exends :term:`pulpcore` to manage one or more
:term:`types<type>` of :term:`content`.
A `Django <https://docs.djangoproject.com>`_ app that exends :term:`pulpcore` to manage one
or more :term:`types<type>` of :term:`content`.

:class:`~pulpcore.app.models.Publication`
The metadata and :term:`artifacts<Artifact>` of the :term:`content units<content>` in a
:term:`repository version<RepositoryVersion>`. Publications are served by the
:term:`content app` when they are assigned to a :term:`distribution`.

:class:`~pulpcore.app.models.Publisher`
A :term:`plugin` defined object that contains settings required to publish a specific :term:`type` of
:term:`content unit<content>`.
A :term:`plugin` defined object that contains settings required to publish a specific
:term:`type` of :term:`content unit<content>`.

pulpcore
A generalized backend with a Plugin API and a :doc:`REST
Expand All @@ -50,7 +55,8 @@ Glossary
A versioned set of :term:`content units<content>`.

:class:`~pulpcore.app.models.RepositoryVersion`
An immutable snapshot of the set of :term:`content units<content>` that are in a :term:`repository`.
An immutable snapshot of the set of :term:`content units<content>` that are in a
:term:`repository`.

sync
A :term:`plugin` defined task that fetches :term:`content` from an external source using a
Expand Down
2 changes: 0 additions & 2 deletions docs/workflows/deferred-download.rst

This file was deleted.

2 changes: 1 addition & 1 deletion docs/workflows/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ assumes that the reader is familiar with the fundamentals discussed in the :doc:
:maxdepth: 2

upload-publish
deferred-download
lazy-downloading
mirroring
promotion
scheduling-tasks
63 changes: 63 additions & 0 deletions docs/workflows/lazy-downloading.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
Lazy Downloading
================

Overview
--------

Pulp can sync content in a few modes: 'immediate', 'on_demand', and 'streamed'. Each provides a
different behavior on how and when Pulp acquires content. These are set as the `policy` attribute
of the :term:`Remote` performing the sync. Policy is an optional parameter and defaults to
`immediate`.

immediate
When performing the sync, download all :term:`Artifacts<artifact>` now. Also download all metadata
now to create the content units in Pulp, associated with the
:term:`repository version<RepositoryVersion>` created by the sync. `immediate` is the default, and
any plugin providing a sync is expected to implement the `immediate` mode.

on_demand
When performing the sync, do not download any :term:`Artifacts<artifact>` now. Download all
metadata now to create the content units in Pulp, associated with the
:term:`repository version<RepositoryVersion>` created by the sync. Clients requesting content
trigger the downloading of :term:`Artifacts<artifact>`, which are saved into Pulp to be served to
future clients.

This mode is ideal for saving disk space because Pulp never downloads and stores
:term:`Artifacts<artifact>` that clients don't need. Units created from this mode are
:term:`lazy content units<lazy content>`.

streamed
When performing the sync, do not download any :term:`Artifacts<artifact>` now. Download all
metadata now to create the content units in Pulp, associated with the
:term:`repository version<RepositoryVersion>` created by the sync. Clients requesting content
trigger the downloading of :term:`Artifacts<artifact>`, which are *not* saved into Pulp. This
content will be re-downloaded with each client request.

This mode is ideal for content that you especially don't want Pulp to store over time. For
instance, syncing from a nightly repo would cause Pulp to store every nightly ever produced which
is likely not valuable. Units created from this mode are :term:`lazy content units<lazy content>`.


Does Plugin X Support Lazy?
---------------------------

Plugins do have to enable support for these modes, so check that plugin's documentation to be sure.
See the `Pulp Plugin API <../../pulpcore-plugin/nightly/>`_ documentation for more details on how to
add lazy support to a plugin.


Associating Lazy Content with Additional Repository Versions
------------------------------------------------------------

A :term:`lazy content unit<lazy content>` can be associated and unassociated from a
:term:`repository version<RepositoryVersion>` just like a normal unit. Note that the original
:term:`Remote` will be used to download content should a client request it, even as that content is
made available in multiple places.


.. warning::

Deleting a :term:`Remote` that was used in a sync with either the `on_demand` or `streamed`
options can break published data. Specifically, clients who want to fetch content that a
:term:`Remote` was providing access to would begin to 404. Recreating a :term:`Remote` and
re-triggering a sync will cause these broken units to recover again.

0 comments on commit 228ae96

Please sign in to comment.