Skip to content

Commit

Permalink
Convert the documentation into a set of .rst files, located in the ne…
Browse files Browse the repository at this point in the history
…w docs/en and docs/fr directories (for respectively the english and the french languages).
  • Loading branch information
beaud76 committed Feb 18, 2017
1 parent b399d31 commit 094d26e
Show file tree
Hide file tree
Showing 96 changed files with 6,075 additions and 0 deletions.
48 changes: 48 additions & 0 deletions docs/en/accessPolicy.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
Set-up the E-Maj access policy
==============================

A bad usage of E-Maj can break the database integrity. So it is advisable to only authorise its use to specific skilled users.

E-Maj roles
-----------

To use E-Maj, it is possible to log on as *superuser*. But for safety reasons, it is preferable to take advantage of both roles created by the installation script:

* **emaj_adm** is used as the administration role ; it can execute all functions and access to all E-Maj tables, with reading and writing rights,
* **emaj_viewer** is used for read only purpose ; it can only execute statistics functions and can only read E-Maj tables.

All rights given to *emaj_viewer* are also given to *emaj_adm*.

When created, these roles have no connection capability (no defined password and *NOLOGIN* option). It is recommended NOT to give them any connection capability. Instead, it is sufficient to give the rights they own to other roles, with *GRANT* SQL verbs.


Giving E-Maj rights
-------------------

Once logged on as *superuser* in order to have the sufficient rights, execute one of the following commands to give a role all rights associated to one of both *emaj_adm* or *emaj_viewer* roles::

GRANT emaj_adm TO <my.emaj.administrator.role>;
GRANT emaj_viewer TO <my.emaj.viewer.role>;

Of course, *emaj_adm* or *emaj_viewer* rights can be given to several roles.


Giving rights on application tables and objects
-----------------------------------------------

To let an E-Maj administrator also access application tables or other application objects (schemas, sequences, views, functions,...), it is possible to give rights on these objects to *emaj_adm* or *emaj_viewer* roles. But it is preferable to only give these rights to the roles which are also given *emaj_adm* or *emaj_viewer* rights, so that the E-Maj roles only directly own rights on E-Maj tables and objects.


Synthesis
---------

The following schema represents the recommended rights organisation for an E-Maj administrator.

.. image:: images/rights.png
:align: center

Of course the schema also applies to *emaj_viewer* role.

Except when explicitly noticed, the operations presented later can be indifferently executed by a *superuser* or by a role belonging to the *emaj_adm* group.


121 changes: 121 additions & 0 deletions docs/en/architecture.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
Architecture
============

In order to be able to perform a rollback operation without having previously kept a physical image of the PostgreSQL cluster's files, all updates applied on application tables must be recorded, so that they can be cancelled.

With E-Maj, this updates recording takes the following form.

Logged SQL statements
*********************
The recorded update operations concerns the following SQL verbs:

* rows insertions:

* INSERT, either elementary (INSERT … VALUES) or set oriented (INSERT … SELECT)
* COPY … FROM

* rows updates:

* UPDATE

* rows deletions:

* DELETE

* tables truncations

* TRUNCATE

For statements that process several rows, each creation, update or deletion is individually recorded. For instance, if a *DELETE FROM <table>* is performed against a table having 1 million rows, 1 million row deletion events are recorded.

The case of *TRUNCATE* SQL verbs is specific. As no *FOR EACH ROW* trigger can be fired for this verb, the consequences of a *TRUNCATE* cannot be cancelled by E-Maj. Therefore, its execution is forbidden for *ROLLBACKABLE* tables groups in *LOGGING* state. In contrast, *TRUNCATE* is always permitted for *AUDIT_ONLY* tables groups. In such a case, only its execution is recorded.


Created objects
***************

For each application table, the following objects are created:

* a dedicated **log table**, containing data corresponding to the updates applied on the application table,
* a **trigger** and a specific **function**, that, for each row creation (*INSERT*, *COPY*), change (*UPDATE*) or suppression (*DELETE*), record into the log table all data needed to potentially cancel later this elementary action,
* another **trigger**, that either blocks any execution of a *TRUNCATE* SQL verb for *ROLLBACKABLE* tables groups or records the execution of a *TRUNCATE* SQL verb for *AUDIT_ONLY* tables groups,
* a **sequence** used to quickly count the number of updates recorded in log tables between 2 marks.

.. image:: images/created_objects.png
:align: center

A **log table** has the same structure as its corresponding application table. However, it contains some additional technical columns:

* a unique identifier, as an integer associated to a global sequence,
* the precise date and time of the update,
* the type of the executed SQL operation: *INS* for *INSERT*, *UPD* for *UPDATE* and *DEL* for *DELETE*,
* an attribute taking either *OLD* or *NEW* value, allowing to distinguish old and new values of updated rows,
* the internal transaction identifier (the PostgreSQL *txid*) that performed the update,
* the connection role who performed the update,
* the ip address of the user who performed the update,
* the ip port of the user who performed the update.

To let E-Maj work, some **other technical objects** are also created at extension installation time:

* 13 tables,
* 5 composite and 2 enum types,
* 1 view,
* more than 90 functions, about half of them being directly callable by users,
* 1 sequence named *emaj_global_seq* used to assign to every update recorded in any log table of the database a unique identifier with an increasing value over time,
* 1 specific schema, named emaj, that contains all these relational objects,
* 2 roles acting as groups (NOLOGIN): *emaj_adm* to manage E-Maj components, and *emaj_viewer* to only look at E-Maj components
* 2 event triggers when the PostgreSQL version is 9.3 or 9.4 and 3 event triggers when the PostgreSQL version is at least 9.5.

Some technical tables, whose structure is interesting to know, are described in details: :ref:`emaj_group_def <emaj_group_def>`, :ref:`emaj_param <emaj_param>` and :ref:`emaj_hist <emaj_hist>`.


Norm for E-Maj objects naming
*****************************

All objects associated to application tables have names built by default with the name of their related table and schema. More precisely, for an application table in a given schema:

* the name of the **log table** is:
<schema.name>_<table.name>_log

* the name of the **log function** is:
<schema.name>_<table.name>_log_fnct

* the name of the **sequence** associated to the log table is:
<schema.name>_<table.name>_log_seq

It is also possible to define for each application table the **prefix** of the associated E-Maj objects name. This allows to manage tables with very long names.

Other E-Maj **function** names are also normalised:

* function names that begin with `emaj_` are functions that are callable by users,
* function names that begin with `_` are internal functions that should not be called directly.

**Triggers** created on application tables have the same name:

* *emaj_log_trg* for the log triggers,
* *emaj_trunc_trg* for the triggers that manage *TRUNCATE* verbs.

The name of **event triggers** starts with `emaj_` and ends with `_trg`.


Schemas
*******

All technical objects created at E-Maj installation are located into the schema named **emaj**.

By default, all objects linked to a tables group are created in the main schema *emaj*. But it is possible to locate these objects in one or several **secondary schemas**. Secondary schemas' names start with `emaj`, only their suffix being parametrized in :ref:`tables groups definition <emaj_group_def>`.

Only one technical object is not located into the emaj schema: the event trigger *emaj_protection_trg* belongs to the public schema.


Tablespaces
***********

E-Maj provides three potential ways to handle **tablespaces**.

When the extension is installed and when log tables are created, E-Maj can use the default tablespace.

But it is also possible to create a dedicated tablespace named *tspemaj*. If it exists when the extension is installed or when a tables group is created, it will be used to hold new tables.

Using :ref:`tables group parameters <emaj_group_def>`, it is also possible to store log tables and/or their index into specific tablespaces.

31 changes: 31 additions & 0 deletions docs/en/concepts.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
Concepts
========

E-Maj is built on three main concepts.

Tables Group
************

The **tables group** represents a set of application tables that live at the same rhythm, meaning that their content can be restored as a whole if needed. Typically, it deals with all tables of a database that are updated by one or more sets of programs. Each tables group is defined by a name which must be unique inside its database. By extent, a tables group can also contain application sequences (in the RDBMS sense). Tables and sequences that constitute a tables group can belong to different schemas of the database.

At a given time, a tables group is either in a **LOGGING** state or in a **IDLE** state. The *LOGGING* state means that all updates applied on the tables of the group are recorded.

A tables group can be either **ROLLBACKABLE**, which is the standard case, or **AUDIT_ONLY**. In this latter case, it is not possible to rollback the group. But with this type of group, it is possible to record tables updates for auditing purposes, even with tables that do not have primary key known in PostgreSQL catalogue.

Mark
****

A **mark** is a particular point in the life of a tables group, corresponding to a stable point for all tables and sequences of the group. A mark is explicitly set by a user operation. It is defined by a name that must be unique for the tables group.

Rollback
********

The **rollback** operation consists of resetting all tables and sequences of a group in the state they had when a mark was set.

There are two rollback types:

* with a **unlogged rollback**, no trace of updates that are cancelled by the rollback operation are kept,
* with a **logged rollback**, update cancellations are recorded in log tables, so that they can be later cancelled: the rollback operation can be … rolled back.

Note that this concept of *E-Maj rollback* is different from the usual concept of *transactions rollback* managed by PostgreSQL.

166 changes: 166 additions & 0 deletions docs/en/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# -*- coding: utf-8 -*-
#
# E-Maj documentation build configuration file, created by
# sphinx-quickstart on Mon Dec 19 19:29:57 2016.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- General configuration ------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = []

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'

# The master toctree document.
master_doc = 'index'

# General information about the project.
project = u'E-Maj'
copyright = u'2017, Philippe Beaudoin'
author = u'Philippe Beaudoin'

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'2.0'
# The full version, including alpha/beta/rc tags.
release = u'2.0.1'

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'

# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False


# -- Options for HTML output ----------------------------------------------

# on_rtd is whether we are on readthedocs.io, this line of code grabbed from docs.readthedocs.io
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'

if not on_rtd: # only import and set the theme if we're building docs locally
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

# otherwise, readthedocs.io uses their theme by default, so no need to specify it

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
#html_theme = 'alabaster'
#html_theme = 'default'

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

# -- Options for HTMLHelp output ------------------------------------------

# Output file base name for HTML help builder.
htmlhelp_basename = 'E-Majdoc'


# -- Options for LaTeX output ---------------------------------------------

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',

# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',

# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',

# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'E-Maj.tex', u'E-Maj Documentation',
u'Philippe Beaudoin', 'manual'),
]


# -- Options for manual page output ---------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'e-maj', u'E-Maj Documentation',
[author], 1)
]


# -- Options for Texinfo output -------------------------------------------

# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'E-Maj', u'E-Maj Documentation',
author, 'E-Maj', 'One line description of project.',
'Miscellaneous'),
]



0 comments on commit 094d26e

Please sign in to comment.