Skip to content

Commit

Permalink
Fix an issue with the sql script generation. When an application tabl…
Browse files Browse the repository at this point in the history 
…e structure has changed and a sql script is generated on a time frame prior the structure change, the current structure of the application table was used, instead of the prior one, leading to wrong scripts. Like for rollback operations, the needed pieces of sql are computed when the table is assigned to its group (in the _create_tbl() function) and recorded into the emaj_relation table. For consistency, some columns of the emaj_relation table have been renamed. Add a regression test in misc.sql for this use case. In passing, when rollbacking a table, the "OVERRIDING SYSTEM VALUE" INSERT clause is only used when needed, sharing with the sql script generation a new flag in the emaj_relation table.
  • Loading branch information
@beaud76
beaud76 committed Dec 12, 2019
1 parent b33a94c commit aba171f
Show file tree
Hide file tree
Showing 59 changed files with 4,787 additions and 3,606 deletions.
2 changes: 2 additions & 0 deletions CHANGES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ E-Maj - Change log
* When tables or sequences were removed from their group or moved to another
group, the internal emaj_sequence table was not properly cleanup at group
reset time or at delete oldest marks time.
* When a table structure has changed, the sql script generation on a time
range prior the structure change produced incorrect statements.


3.2.0 (2019-Oct-15)
Expand Down
1,287 changes: 1,133 additions & 154 deletions sql/emaj--3.2.0--devel.sql
View file

Large diffs are not rendered by default.

336 changes: 164 additions & 172 deletions sql/emaj--devel.sql
View file

Large diffs are not rendered by default.

336 changes: 164 additions & 172 deletions sql/emaj-devel.sql
View file

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions test/10/expected/adm1.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -193,8 +193,8 @@ select sch_name from emaj.emaj_schema;
(1 row)

select * from emaj.emaj_relation;
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_sql_columns | rel_sql_pk_columns | rel_sql_pk_eq_conditions | rel_log_seq_last_value | rel_emaj_verb_attnum
------------+------------+----------------+-----------+----------+--------------+----------------+---------------+-----------------+---------------+-----------------+------------------+------------------+-----------------+--------------------+--------------------------+------------------------+----------------------
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_emaj_verb_attnum | rel_has_always_ident_col | rel_sql_rlbk_columns | rel_sql_rlbk_pk_columns | rel_sql_rlbk_pk_conditions | rel_sql_gen_ins_col | rel_sql_gen_ins_val | rel_sql_gen_upd_set | rel_sql_gen_pk_conditions | rel_log_seq_last_value
------------+------------+----------------+-----------+----------+--------------+----------------+---------------+-----------------+---------------+-----------------+------------------+------------------+----------------------+--------------------------+----------------------+-------------------------+----------------------------+---------------------+---------------------+---------------------+---------------------------+------------------------
(0 rows)

select * from emaj.emaj_mark;
Expand Down
44 changes: 24 additions & 20 deletions test/10/expected/adm2.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2270,12 +2270,14 @@ select stat_group, stat_schema, stat_table, stat_first_mark, stat_last_mark, sta
myGroup4 | myschema4 | mytblm | Start | | 0
(6 rows)

select * from emaj.emaj_relation where rel_schema = 'myschema4' and rel_tblseq like 'mypar%' order by rel_tblseq, rel_time_range;
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_sql_columns | rel_sql_pk_columns | rel_sql_pk_eq_conditions | rel_log_seq_last_value | rel_emaj_verb_attnum
------------+------------+----------------+-----------+----------+--------------+----------------+----------------+-----------------+--------------------+-----------------+------------------+-------------------+----------------------------+--------------------+--------------------------+------------------------+----------------------
myschema4 | mypartp1 | [10099,10111) | myGroup4 | r | | emaj_myschema4 | mypartp1_log_1 | | mypartp1_log_idx_1 | | mypartp1_log_seq | | | | | 1 | 4
myschema4 | mypartp2 | [10099,) | myGroup4 | r | | emaj_myschema4 | mypartp2_log | | mypartp2_log_idx | | mypartp2_log_seq | mypartp2_log_fnct | tbl.col1,tbl.col2,tbl.col3 | col1 | tbl.col1 = keys.col1 | | 4
myschema4 | mypartp3 | [10110,) | myGroup4 | r | | emaj_myschema4 | mypartp3_log | | mypartp3_log_idx | | mypartp3_log_seq | mypartp3_log_fnct | tbl.col1,tbl.col2,tbl.col3 | col1 | tbl.col1 = keys.col1 | | 4
select rel_schema, rel_tblseq, rel_time_range, rel_group, rel_kind, rel_log_schema, rel_log_table,
rel_emaj_verb_attnum, rel_has_always_ident_col, rel_log_seq_last_value
from emaj.emaj_relation where rel_schema = 'myschema4' and rel_tblseq like 'mypar%' order by rel_tblseq, rel_time_range;
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_log_schema | rel_log_table | rel_emaj_verb_attnum | rel_has_always_ident_col | rel_log_seq_last_value
------------+------------+----------------+-----------+----------+----------------+----------------+----------------------+--------------------------+------------------------
myschema4 | mypartp1 | [10099,10111) | myGroup4 | r | emaj_myschema4 | mypartp1_log_1 | 4 | f | 1
myschema4 | mypartp2 | [10099,) | myGroup4 | r | emaj_myschema4 | mypartp2_log | 4 | f |
myschema4 | mypartp3 | [10110,) | myGroup4 | r | emaj_myschema4 | mypartp3_log | 4 | f |
(3 rows)

select col1, col2, emaj_verb, emaj_tuple, emaj_gid from emaj_myschema4.mypartP3_log order by emaj_gid;
Expand Down Expand Up @@ -2455,10 +2457,10 @@ NOTICE: trigger "emaj_trunc_trg" for relation "phil's schema3.mytbl4" does not
(1 row)

select * from emaj.emaj_relation where rel_schema = 'phil''s schema3' and rel_tblseq = 'mytbl4' order by rel_time_range;
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_sql_columns | rel_sql_pk_columns | rel_sql_pk_eq_conditions | rel_log_seq_last_value | rel_emaj_verb_attnum
----------------+------------+----------------+------------------+----------+--------------+---------------------+---------------+-----------------+------------------+-----------------+------------------+------------------+---------------------------------------------------+--------------------+--------------------------+------------------------+----------------------
phil's schema3 | mytbl4 | [10118,10120) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log_1 | | mytbl4_log_idx_1 | | mytbl4_log_seq | | | | | 0 | 6
phil's schema3 | mytbl4 | [10121,) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log | | mytbl4_log_idx | | mytbl4_log_seq | mytbl4_log_fnct | tbl.col41,tbl.col42,tbl.col43,tbl.col44,tbl.col45 | col41 | tbl.col41 = keys.col41 | | 6
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_emaj_verb_attnum | rel_has_always_ident_col | rel_sql_rlbk_columns | rel_sql_rlbk_pk_columns | rel_sql_rlbk_pk_conditions | rel_sql_gen_ins_col | rel_sql_gen_ins_val | rel_sql_gen_upd_set | rel_sql_gen_pk_conditions | rel_log_seq_last_value
----------------+------------+----------------+------------------+----------+--------------+---------------------+---------------+-----------------+------------------+-----------------+------------------+------------------+----------------------+--------------------------+---------------------------------------------------+-------------------------+----------------------------+---------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+------------------------
phil's schema3 | mytbl4 | [10118,10120) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log_1 | | mytbl4_log_idx_1 | | mytbl4_log_seq | | 6 | f | | | | | ' || coalesce(o.col41::TEXT,'NULL') || ', ' || quote_nullable(o.col42) || ', ' || coalesce(o.col43::TEXT,'NULL') || ', ' || coalesce(o.col44::TEXT,'NULL') || ', ' || quote_nullable(o.col45) || ' | col41 = ' || coalesce(n.col41 ::TEXT,'NULL') || ', col42 = ' || quote_nullable(n.col42) || ', col43 = ' || coalesce(n.col43 ::TEXT,'NULL') || ', col44 = ' || coalesce(n.col44 ::TEXT,'NULL') || ', col45 = ' || quote_nullable(n.col45) || ' | col41 = ' || o.col41 || ' | 0
phil's schema3 | mytbl4 | [10121,) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log | | mytbl4_log_idx | | mytbl4_log_seq | mytbl4_log_fnct | 6 | f | tbl.col41,tbl.col42,tbl.col43,tbl.col44,tbl.col45 | col41 | tbl.col41 = keys.col41 | | ' || coalesce(o.col41::TEXT,'NULL') || ', ' || quote_nullable(o.col42) || ', ' || coalesce(o.col43::TEXT,'NULL') || ', ' || coalesce(o.col44::TEXT,'NULL') || ', ' || quote_nullable(o.col45) || ' | col41 = ' || coalesce(n.col41 ::TEXT,'NULL') || ', col42 = ' || quote_nullable(n.col42) || ', col43 = ' || coalesce(n.col43 ::TEXT,'NULL') || ', col44 = ' || coalesce(n.col44 ::TEXT,'NULL') || ', col45 = ' || quote_nullable(n.col45) || ' | col41 = ' || o.col41 || ' |
(2 rows)

-----------------------------
Expand Down Expand Up @@ -2495,12 +2497,13 @@ NOTICE: trigger "emaj_trunc_trg" for relation "phil's schema3.mytbl4" does not
7
(1 row)

select * from emaj.emaj_relation where rel_schema = 'phil''s schema3' and rel_tblseq = 'mytbl4' order by rel_time_range;
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_sql_columns | rel_sql_pk_columns | rel_sql_pk_eq_conditions | rel_log_seq_last_value | rel_emaj_verb_attnum
----------------+------------+----------------+------------------+----------+--------------+---------------------+---------------+-----------------+------------------+-----------------+------------------+------------------+---------------------------------------------------+--------------------+--------------------------+------------------------+----------------------
phil's schema3 | mytbl4 | [10118,10120) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log_1 | | mytbl4_log_idx_1 | | mytbl4_log_seq | | | | | 0 | 6
phil's schema3 | mytbl4 | [10121,10122) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log_2 | | mytbl4_log_idx_2 | | mytbl4_log_seq | | | | | 0 | 6
phil's schema3 | mytbl4 | [10123,) | phil's group#3", | r | | emaj_phil's schema3 | mytbl4_log | | mytbl4_log_idx | | mytbl4_log_seq | mytbl4_log_fnct | tbl.col41,tbl.col42,tbl.col43,tbl.col44,tbl.col45 | col41 | tbl.col41 = keys.col41 | | 6
select rel_schema, rel_tblseq, rel_time_range, rel_group, rel_kind
from emaj.emaj_relation where rel_schema = 'phil''s schema3' and rel_tblseq = 'mytbl4' order by rel_time_range;
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind
----------------+------------+----------------+------------------+----------
phil's schema3 | mytbl4 | [10118,10120) | phil's group#3", | r
phil's schema3 | mytbl4 | [10121,10122) | phil's group#3", | r
phil's schema3 | mytbl4 | [10123,) | phil's group#3", | r
(3 rows)

-----------------------------
Expand Down Expand Up @@ -3192,10 +3195,11 @@ select * from emaj.emaj_rel_hist order by 1,2,3;
phil's schema3 | table_with_very_looooooooooooooooooooooooooooooooooooooong_name | [10061,10066) | phil's group#3", | r
(57 rows)

select * from emaj.emaj_relation where rel_schema in ('phil''s schema3','myschema4');
rel_schema | rel_tblseq | rel_time_range | rel_group | rel_kind | rel_priority | rel_log_schema | rel_log_table | rel_log_dat_tsp | rel_log_index | rel_log_idx_tsp | rel_log_sequence | rel_log_function | rel_sql_columns | rel_sql_pk_columns | rel_sql_pk_eq_conditions | rel_log_seq_last_value | rel_emaj_verb_attnum
------------+------------+----------------+-----------+----------+--------------+----------------+---------------+-----------------+---------------+-----------------+------------------+------------------+-----------------+--------------------+--------------------------+------------------------+----------------------
(0 rows)
select count(*) from emaj.emaj_relation where rel_schema in ('phil''s schema3','myschema4');
count
-------
0
(1 row)

-----------------------------
-- test end: check, reset history and force sequences id
Expand Down

0 comments on commit aba171f

Please sign in to comment.