Skip to content

Commit

Permalink
Documentation: Several minor improvements. Also document the dblink_u…
Browse files Browse the repository at this point in the history
…ser_password parameter that was missing in the parameters list.
  • Loading branch information
beaud76 committed Sep 19, 2020
1 parent 1a0f449 commit f4b759b
Show file tree
Hide file tree
Showing 10 changed files with 148 additions and 141 deletions.
136 changes: 68 additions & 68 deletions docs/en/extractionFunctions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,74 @@ Data extraction functions

Three functions extract data from E-Maj infrastructure and store them into external files.

.. _emaj_gen_sql_group:

SQL script generation to replay logged updates
----------------------------------------------

Log tables contain all needed information to replay updates. Therefore, it is possible to generate SQL statements corresponding to all updates that occurred between two marks or between a mark and the current situation. This is the purpose of the *emaj_gen_sql_group()* function.

So these updates can be replayed after the corresponding tables have been restored in their state at the initial mark, without being obliged to rerun application programs.

To generate this SQL script, just execute the following statement::

SELECT emaj.emaj_gen_sql_group('<group.name>', '<start.mark>', '<end.mark>', '<file>' [, <tables/sequences.array>);

A *NULL* value or an empty string may be used as start mark, representing the first known mark.

A *NULL* value or an empty string may be used as end mark, representing the current situation.

The keyword *'EMAJ_LAST_MARK'* can be used as mark name, representing the last set mark.

If supplied, the output file name must be an absolute pathname. It must have the appropriate permission so that the PostgreSQL instance can write to it. If the file already exists, its content is overwritten.

The output file name may be set to NULL. In this case, the SQL script is prepared in a temporary table that can then be accessed through a temporary view, *emaj_sql_script*. Using *psql*, the script can be exported with both commands::

SELECT emaj.emaj_gen_sql_group('<group.name>', '<start.mark>', '<end.mark>', NULL [, <tables/sequences.array>);
\copy (SELECT * FROM emaj_sql_script) TO ‘file’

This method allows to generate a script in a file located outside the file systems accessible by the PostgreSQL instance.

The last parameter of the *emaj_gen_sql_group()* function is optional. It allows filtering of the tables and sequences to process. If the parameter is omitted or has a *NULL* value, all tables and sequences of the tables group are processed. If specified, the parameter must be expressed as a non empty array of text elements, each of them representing a schema qualified table or sequence name. Both syntaxes can be used::

ARRAY['sch1.tbl1','sch1.tbl2']

or::

'{ "sch1.tbl1" , "sch1.tbl2" }'

The function returns the number of generated statements (not including comments and transaction management statements).

The tables group may be in *IDLE* or in *LOGGING* state while the function is called.

In order to generate the script, all tables must have an explicit *PRIMARY KEY*.

.. caution::

If a tables and sequences list is specified to limit the *emaj_gen_sql_group()* function's work, it is the user's responsibility to take into account the possible presence of foreign keys, in order to let the function produce a viable SQL script.

Statements are generated in the order of their initial execution.

The statements are inserted into a single transaction. They are surrounded by a *BEGIN TRANSACTION;* statement and a *COMMIT;* statement. An initial comment specifies the characteristics of the script generation: generation date and time, related tables group and used marks.

At the end of the script, sequences belonging to the tables group are set to their final state.

Then, the generated file may be executed as is by psql tool, using a connection role that has enough rights on accessed tables and sequences.

The used technology may result to doubled backslashes in the output file. These doubled characters must be suppressed before executing the script, for instance, in Unix/Linux environment, using a command like::

sed 's/\\\\/\\/g' <file.name> | psql ...

As the function can generate a large, or even very large, file (depending on the log volume), it is the user's responsibility to provide a sufficient disk space.

It is also the user's responsibility to deactivate application triggers, if any exist, before executing the generated script.

Using the *emaj_gen_sql_groups()* function, it is possible to generate a sql script related to several groups::

SELECT emaj.emaj_gen_sql_groups('<group.names.array>', '<start.mark>', '<end.mark>', '<file>' [, <tables/sequences.array>);

More information about :doc:`multi-groups functions <multiGroupsFunctions>`.

.. _emaj_snap_group:

Snap tables of a group
Expand Down Expand Up @@ -83,71 +151,3 @@ As this function may generate large or very large files (of course depending on
The structure of log tables is directly derived from the structure of the related application table. The log tables contain the same columns with the same type. But they also have some additional technical columns:

The structure of log tables is described :ref:`here <logTableStructure>`.

.. _emaj_gen_sql_group:

SQL script generation to replay logged updates
----------------------------------------------

Log tables contain all needed information to replay updates. Therefore, it is possible to generate SQL statements corresponding to all updates that occurred between two marks or between a mark and the current situation. This is the purpose of the *emaj_gen_sql_group()* function.

So these updates can be replayed after the corresponding tables have been restored in their state at the initial mark, without being obliged to rerun application programs.

To generate this SQL script, just execute the following statement::

SELECT emaj.emaj_gen_sql_group('<group.name>', '<start.mark>', '<end.mark>', '<file>' [, <tables/sequences.array>);

A *NULL* value or an empty string may be used as start mark, representing the first known mark.

A *NULL* value or an empty string may be used as end mark, representing the current situation.

The keyword *'EMAJ_LAST_MARK'* can be used as mark name, representing the last set mark.

If supplied, the output file name must be an absolute pathname. It must have the appropriate permission so that the PostgreSQL instance can write to it. If the file already exists, its content is overwritten.

The output file name may be set to NULL. In this case, the SQL script is prepared in a temporary table that can then be accessed through a temporary view, *emaj_sql_script*. Using *psql*, the script can be exported with both commands::

SELECT emaj.emaj_gen_sql_group('<group.name>', '<start.mark>', '<end.mark>', NULL [, <tables/sequences.array>);
\copy (SELECT * FROM emaj_sql_script) TO ‘file’

This method allows to generate a script in a file located outside the file systems accessible by the PostgreSQL instance.

The last parameter of the *emaj_gen_sql_group()* function is optional. It allows filtering of the tables and sequences to process. If the parameter is omitted or has a *NULL* value, all tables and sequences of the tables group are processed. If specified, the parameter must be expressed as a non empty array of text elements, each of them representing a schema qualified table or sequence name. Both syntaxes can be used::

ARRAY['sch1.tbl1','sch1.tbl2']

or::

'{ "sch1.tbl1" , "sch1.tbl2" }'

The function returns the number of generated statements (not including comments and transaction management statements).

The tables group may be in *IDLE* or in *LOGGING* state while the function is called.

In order to generate the script, all tables must have an explicit *PRIMARY KEY*.

.. caution::

If a tables and sequences list is specified to limit the *emaj_gen_sql_group()* function's work, it is the user's responsibility to take into account the possible presence of foreign keys, in order to let the function produce a viable SQL script.

Statements are generated in the order of their initial execution.

The statements are inserted into a single transaction. They are surrounded by a *BEGIN TRANSACTION;* statement and a *COMMIT;* statement. An initial comment specifies the characteristics of the script generation: generation date and time, related tables group and used marks.

At the end of the script, sequences belonging to the tables group are set to their final state.

Then, the generated file may be executed as is by psql tool, using a connection role that has enough rights on accessed tables and sequences.

The used technology may result to doubled backslashes in the output file. These doubled characters must be suppressed before executing the script, for instance, in Unix/Linux environment, using a command like::

sed 's/\\\\/\\/g' <file.name> | psql ...

As the function can generate a large, or even very large, file (depending on the log volume), it is the user's responsibility to provide a sufficient disk space.

It is also the user's responsibility to deactivate application triggers, if any exist, before executing the generated script.

Using the *emaj_gen_sql_groups()* function, it is possible to generate a sql script related to several groups::

SELECT emaj.emaj_gen_sql_groups('<group.names.array>', '<start.mark>', '<end.mark>', '<file>' [, <tables/sequences.array>);

More information about :doc:`multi-groups functions <multiGroupsFunctions>`.
3 changes: 2 additions & 1 deletion docs/en/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ E-Maj and its add-ons are also available on the **github.org** Internet site:
* documentation (https://github.com/beaud76/emaj_doc)
* Emaj_web GUI (https://github.com/dalibo/emaj_web)

For the records, an old repository (https://github.com/beaud76/emaj_ppa_plugin) contains a plugin for phpPgAdmin. It is not maintained since E-Maj 3.0.
.. caution::
Installing the extension from the *github.org* repository creates the extension in its development version ("devel"). In this case, no future extension update is possible. For a stable E-Maj use, it is highly recommended to use the packets available from *PGXN*.

Standart installation on Linux
******************************
Expand Down
2 changes: 1 addition & 1 deletion docs/en/parallelRollbackClient.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ To allow the rollback operation to work, the tables group or groups must be in *

The *'EMAJ_LAST_MARK'* keyword can be used as mark name, meaning the last set mark.

It is possible to monitor the multi-session rollback operations with the same tools as for mono-session rollbacks.
It is possible to monitor the multi-session rollback operations with the same tools as for mono-session rollbacks: :ref:`emaj_rollback_activity()<emaj_rollback_activity>` function, the :doc:`emajRollbackMonitor<rollbackMonitorClient>` command or the Emaj_web rollback monitor page. As for mono-session rollbacks, the :doc:`dblink_user_password<parameters>` parameter must be set in order to get detailed status of the operations progress.

In order to test both *emajParallelRollback* commands, the E-Maj extension supplies a test script, *emaj_prepare_parallel_rollback_test.sql*. It prepares an environment with two tables groups containing some tables and sequences, on which some updates have been performed, with intermediate marks. Once this script has been executed under *psql*, the command displayed at the end of the script can be simply run.

Expand Down
1 change: 1 addition & 0 deletions docs/en/parameters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Presented in alphabetic order, the existing key values are:
* **avg_fkey_check_duration** : (interval) default value = 20 µs ; defines the average duration of a foreign key value check ; can be modified to better represent the performance of the server that hosts the database when using the :ref:`emaj_estimate_rollback_group() <emaj_estimate_rollback_group>` function.
* **avg_row_delete_log_duration** : (interval) default value = 10 µs ; defines the average duration of a log row deletion ; can be modified to better represent the performance of the server that hosts the database when using the :ref:`emaj_estimate_rollback_group() <emaj_estimate_rollback_group>` function.
* **avg_row_rollback_duration** : (interval) default value = 100 µs ; defines the average duration of a row rollback ; can be modified to better represent the performance of the server that hosts the database when using the :ref:`emaj_estimate_rollback_group() <emaj_estimate_rollback_group>` function.
* **dblink_user_password** : (text) empty string by default ; format = 'user=<user> password=<password>' ; defines the user and password that elementary functions executing :ref:`E-Maj rollback operations<emaj_rollback_group>` can use to update the internal rollback monitoring tables with autonomous transactions. This is required to monitor the in progress E-Maj rollback operations.
* **fixed_dblink_rollback_duration** : (interval) default value = 4 ms ; defines an additional cost for each rollback step when a dblink connection is used ; can be modified to better represent the performance of the server that hosts the database when using the :ref:`emaj_estimate_rollback_group() <emaj_estimate_rollback_group>` function.
* **fixed_table_rollback_duration** : (interval) default value = 1 ms ; defines a fixed rollback cost for any table belonging to a group ; can be modified to better represent the performance of the server that hosts the database when using the :ref:`emaj_estimate_rollback_group() <emaj_estimate_rollback_group>` function.
* **fixed_step_rollback_duration** : (interval) default value = 2,5 ms ; defines a fixed cost for each rollback step ; can be modified to better represent the performance of the server that hosts the database when using the :ref:`emaj_estimate_rollback_group() <emaj_estimate_rollback_group>` function.
Expand Down
2 changes: 2 additions & 0 deletions docs/en/rollbackMonitorClient.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ Two equivalent tools are actually provided, one coded in *php* and the other in
* for the *php* client, the **php** software and its PostgreSQL interface,
* for the *perl* client, the **perl** software with the *DBI* and *DBD::Pg* modules.

In order to get detailed information about the in-progress rollback operations, it is necessary to set the :doc:`dblink_user_password<parameters>` parameter.

Syntax
------

Expand Down

0 comments on commit f4b759b

Please sign in to comment.