forked from ssinger/slony1-engine
/
RELEASE
591 lines (407 loc) · 22.1 KB
/
RELEASE
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
$Id: RELEASE,v 1.1.2.33 2008-09-12 17:37:48 cbbrowne Exp $
Release 1.2.15
- Fix to STORE TRIGGER - store trigger was running against all nodes
upon subscription
http://www.slony.info/bugzilla/show_bug.cgi?id=56
- Portability changes to some tools/ scripts, fixing some
"bash-isms"
- Fix to switch statement in slonik.c; unknown how it broke
- Fix memory leak
http://bugs.slony.info/bugzilla/show_bug.cgi?id=52
RELEASE 1.2.14
- Fix typo in configure-replication.sh (missing CR)
- Per bug #35, search the Slony share dir for scripts
before falling back to the PG share dir on 8.0+
http://www.slony.info/bugzilla/show_bug.cgi?id=35
- Change test framework to write out the test name into
$TEMPDIR/TestName
- Patch that seems to resolve a race condition with
ACCEPT_SET
http://lists.slony.info/pipermail/slony1-general/2008-March/007655.html
- Fix bug #49 - mishandling by slony_logshipper of quotes &
backslashes.
- Fix bug #50 - slony_logshipper had a variable access *after*
memory was freed
RELEASE 1.2.13
- Fixed problem with compatibility with PostgreSQL 8.3; function
typenameTypeId() has 3 arguments as of 8.3.
- Added in logic to ensure that max # of SYNCs grouped together is
actually constrained by config parameter sync_group_maxsize.
- Fix to show_slony_configuration - point to proper directory where
slon/slonik are actually installed.
- Fix to slonik Makefile + slonik.c - Change slonik build to query
Postgres for the share directory at runtime - per Dave Page
- Removed spurious NOTIFY on "_%s_Confirm"; this is no longer needed
in the 1.2 branch, as there is no LISTEN on this notification.
Noted in bug #32 - http://www.slony.info/bugzilla/show_bug.cgi?id=32
RELEASE 1.2.12
- Fixed problem with DDL SCRIPT parser where C-style comments were
not being processed properly
- Added stored functions and documentation for adding empty tables
(notably *partitions*) to replication. Note these functions
do no work when not specifically requested.
CAVEAT: This functionality may not work as expected on versions
of PostgreSQL earlier than 8.1. Mind you, partitioning tends
to function pretty poorly in earlier versions of PostgreSQL as
there were substantial enhancements in 8.1 and following versions.
- Added a fairly substantial partitioning test to exercise the
new stored functions above.
- Backport "listen path" generator function from CVS HEAD (2.0) to
1.2 branch.
- Fixed a problem with "EXECUTE SCRIPT" (introduced in remote_worker.c
version 1.124.2.13) where moving the relevant code into a subroutine
at the end led to losing the "BEGIN; SET TRANSACTION ISOLATION LEVEL
SERIALIZABLE;" query that needs to be the first thing run...
- Fixing the archive sequence generations (in log shipping). All
non-SYNC events must start the local transaction before creating the
archive as well, so that the lock on the archive counter table
serializes archive creation.
- Fixed logging done in local_listener.c - various places, there was
no '\n' in some cases, which would lead to entries being folded
together.
- Fix launch_slons.sh - was not stripping quotes from PID file name
- Error handling for "ERROR: could not serialize access due to
concurrent update"
If this error is encountered when starting processing of
sl_archive_counter, then two threads are fighting over access to
this counter, and at least one has just failed.
Rather than waiting, we ask to restart the node immediately.
- Fixes to slonik_build_env script - it wasn't properly handling
cases where there was just 1 table or 1 sequence, and had a
problem with the -schema option - thanks, Bernd Helmle
- Don't bother building slony_logshipper on Win32 as it doesn't work
there at this point.
- If slonik connects as other than a superuser, then generate error
message indicating this to the user.
RELEASE 1.2.11
- Add in tools/mkservice scripts previously added to CVS HEAD
- During subscription, do UPDATE to pg_class.relhasindex *after* the
TRUNCATE because, in 8.2+, TRUNCATE resets this attribute
- Fixed a problem with the setsync tracking with Log Shipping in cases
where slon does an internal restart (thereby rereading the
pset.ssy_seqno) and ignoring non-SYNC events because those don't
change the sl_setsync table.
- More explicit type casting of text objects for compatibility with
PostgreSQL 8.3
- Fixed problem with DDL SCRIPT statement parser: it wasn't 'quoting'
semicolons inside parentheses (this notably occurs in CREATE RULE).
- Fixed problem with DDL SCRIPT statement submission; it was
interpreting the statement as a format string, which would have
ill effects in the presence of things that are interpreted such
as format strings (%d, %f, %s) and \backslashed things like \\, \n.
- Further DDL Script issue: non-terminated statement at the end
(e.g. - without trailing semicolon ";") would get omitted.
- Added tests to the log shipping case to check for
invalidly-evaluated printf() format strings; added data that
contains such data, and prepared a DDL script that contained gory
format strings.
- Typo fix: when trying to disable a node, the logs would report
"enableNode" rather than "disableNode". Fixed.
- Add usage/version options to help output in slon.
- Fix archive logging for replicated sequences.
- Fix to log shipping - added another table, sl_archive_counter where
the log writing slon simply tracks when it wrote the last offline
archive file and maintains a counter. This counter is now tracked in
the offline replica and must increment gap free.
- Change the filenames of archive logs to be based on internal archive
tracking number. This makes it easy for the mechanism applying
archives to figure out what needs to be applied next - just look in
sl_archive_tracking.
- Fix log shipping test to accomodate the new tracking scheme, and
update documentation to describe this better.
RELEASE 1.2.10
- Fixed problem with EXECUTE SCRIPT (EXECUTE ONLY ON = <node>)
- The script was being executed on too many nodes...
- Added a test script for log shipping
... And alter it to add invocation of a DDL script. This
allows testing for an event-counting problem in log shipping.
- Changes to support PostgreSQL 8.3 as VARATT_SIZEP has been deprecated
- in xxid.c, changes to support PostgreSQL 8.3 as tuple compression has
more extensive support
VARDATA_ANY(PG_DETOAST_DATUM_PACKED((PG_GETARG_DATUM(0))))
tends to replace
PG_GETARG_VARLENA(0)
- Slonik's SYNC command never recorded the new seqno in the admin
conninfo, with potential wacky results for WAIT FOR EVENT
- Fix rpm build problem when the system has pg_config in both under
/usr/local/pgsql/bin and /usr/bin
- Add init script for Red Hat / Fedora
- Fix archive log ship tracking. Slon now tracks the setsync status in
memory and generates a void archive with the correct old,new event
seqno for all events.
- EXECUTE SCRIPT wasn't setting sl_setsync, which broke WAIT FOR EVENT
on these events
- Ducttape test #5 (which performs DDL changes) augmented to test use
of WAIT FOR EVENT
- Backpatch fixes for restricted text casts in 8.3
create_event() calls (notably, but they're not alone) need to be
augmented to cast data types expressly.
- PostgreSQL 8.3 is now a supported version for 1.2.10
The hope/intent is that 1.2.10 will be the last version of Slony-I
in the "1.x" series, and will permit upgrades to PostgreSQL 8.3.
The subsequent Slony-I 2.x series will require PostgreSQL 8.3.
- Discovered and documented a problem with UPDATE FUNCTIONS on
versions 8.1.[0-3].
- Fixes to log shipping test
- Win32 fix for MinGW+gcc
- cvs distclean changed to clean the build up more...
- On copy_set() check that we have forwarded the confirmation of
ENABLE_SUBSCRIPTION by the data provider only if the data provider
is not the origin of the set.
- For logshipping we need to use the internally tracked ssy_seqno for
SYNC events as well.
This allows "run_test.sh testlogship" to run successfully!
RELEASE 1.2.9
- Reverted change that tried to support elderly apache rotatelogs
- Added a patch file to apply if you need to support elderly apache rotatelogs (tools/altperl/old-apache-rotatelogs.patch)
- Bug in UPDATE FUNCTIONS - wrong quoting in plpgsql function
- Add a regression test that runs UPDATE FUNCTIONS to ensure that it at least has no syntax errors
RELEASE 1.2.8
- Change to rotatelogs configuration to support older versions of Apache
log rotator
- Fix to altperl "execute script" script to pass the filename properly
- Fix to src/backend/slony1_funcs.v80.sql - comment on the right function in v8.0
- Fix to src/slonik/slonik.c - it wasn't pulling in the right version of slony1_funcs.?.sql in some cases
- Updated docs on creating releases to describe the version mismatch problem found above in slonik.c
RELEASE 1.2.7
- Add remote_listen_timeout parameter to slon.conf
This addresses the problem where a slon times out when accessing
sl_event if a node has been out of commission for a long time (several
days)
- Resolve bug #1623
In this bug, big "action lists" that need to get compressed could cause
a logging printf to blow up. Changed the logging level so that detail
is only shown at level 4, which won't bite people by default.
- UNINSTALL NODE failures now show node # in slonik error messages
If a user ran several UNINSTALL NODE requests in a single slonik
script, and one of them broke, you'd have no ready way to tell which
node this failed on. Added code to report the node # where it failed.
- Added test to test1 for function generate_sync_event() and make_function_strict
- Added "v81" files (for slony1_base.v81.sql, slony1_funcs.v81.sql,
xxid.v81.sql), necessary to support 8.1 "ALTER FUNCTION ... STRICT";
- Fixed quoting problem in generate_sync_event()
- Added functionality to UPDATE FUNCTIONS to make xxidin() function
STRICT; the absence of this caused postmaster to fall over when
processing MOVE SET event in PG v8.2
- Added documentation of an issue surrounding NULLABLE columns to the
log analysis chapter of the admin guide and to the UPGRADING docs.
- When you run MOVE SET, this populates sl_setsync for the moved set
even on nodes that are not subscribed. If, subsequent to doing this,
you attempt a SUBSCRIBE SET for a formerly-unsubscribed node, the
subscription will fail right at the end when the slon tries to insert a
new value to sl_setsync.
The fix: DELETE from sl_setsync immediately before the INSERT. This
will silently blow away any 'offending' sl_setsync row.
(As observed by Afilias staff...)
- Log shipping fix - storage of sl_setsync_offline call had a wrong
printf type; change from %d to %s
RELEASE 1.2.6
Release 1.2.6 fixes version numbers.
RELEASE 1.2.5
Release 1.2.5 of Slony-I has the following bug fixes over 1.2.2:
- Fixed issue with xxid C functions - these functions need to be
defined STRICT so that NULL values don't cause the functions to
barf. This has been causing users of PostgreSQL 8.2 to see
postmasters falling over.
- Several improvements to internal documentation of altperl scripts
RELEASE 1.2.2
Release 1.2.2 of Slony-I has the following bug fixes over 1.2.1:
- SGML tagging fixes to allow docs to be generated on Fedora
- Fixes to altperl scripts (init cluster, store node) so they would
properly generate STORE PATH requests (which had broken when STORE
LISTEN code was removed)
- If PostgreSQL version is 8.1 or greater, we can use pg_config
sharedir. Otherwise we will resort to the default guessing mode.
- Some makefile hygenic fixes
- Added \n to a number of log requests
- Fix to UPDATE FUNCTIONS which did not consider versions 1.1.5/1.1.6
to be members of the 1.1 stream. The minor problem with this was
that Slony-I tables wouldn't get altered to drop out OIDs. The
major problem was that the new table sl_registry wouldn't get
created
- Removed some confusing NOTICEs concerning management of partial
indexes on sl_log_1 and sl_log_2.
- Close file descriptors upon running slon_terminate_worker() so that
we don't run out of file descriptors
- Added in sleep(10) requests in several places where attempts to
access the database in the main thread fails; this means that, for
instance, the main DB is down, the slon doesn't simply sit there
continually attempting to reconnect, as many times per second as it
can.
- Several fixes to RPM .spec files
- Environment fix to Win32 service
- Set escape_string_warning to off, to prevent pgsql log noise when
replicating cross versions
- Fix memory leak: free logshipping query
- Bug #1585 - on 8.0+, if TRUNCATE failed, indices were not being
deactivated during SUBSCRIBE SET COPY request, greatly slowing loading
of data. Added a deactivation inside the exception block, so that
indices are deactivated during the COPY no matter what.
- X-Fade noticed that UPGRADE FUNCTIONS was taking out exclusive locks
on sl_log_1, sl_log_2, sl_seqlog, which means that doing a Slony-I
upgrade involved imposing an application outage.
Moved the ALTER TABLE ... WITHOUT OIDS requests for these tables to
cleanup loops, elsewhere, so that these tables no longer need to be
locked as part of the upgrade process.
- Documentation changes: Need for client/server encodings to match
- Fixes to parameter handling and interpreter name for
test_slony_state*.pl scripts
- There was an error condition where if the final line of a slon.conf
file did not end with a \n, then:
a) A comment on that line would be treated as a syntax error
b) A value set on that line would be ignored
Fixed.
RELEASE 1.2.1
Release 1.2.1 of Slony-I has the following improvements over 1.2.0:
- Added a SYNC slonik command to allow requesting a SYNC event.
This is a more useful thing to wait for than a SUBSCRIBE SET.
- MERGE SET now refuses to be applied if a subscriber has not
yet confirmed post-subscription events
- Added a SLEEP slonik command to allow writing code that tells
the script to pause for some number of seconds
- Fixed a concurrency problem where if you had multiple log
shipped nodes, the threads processing them shared a single
file descriptor as well as other variables.
- If a child process exits with return code of 0, that means it
*asked* to exit, presumably because configuration changed that it
needs to reload. In that case, there's no need to wait 10s before
restarting the thread; we should restart the thread immediately.
This causes operations like MOVE SET to run ~10s faster per
subscription change.
- Added a script, tools/release_checklist.sh, which performs some of
the release checklist checks, such as verifying that the Slony-I
version number (e.g. - 1.2.1) is consistent across various places
that need to reference it.
- altperl scripts have had "listen path" generation removed as that is
handled automatically as clusters are built.
- Fixed some portability problems in tools/launch_cluster.sh
- Added a new script, tools/search_logs.sh, meant to search hourly for
errors in slon logs and warn an administrator about them.
RELEASE 1.2.0
Release 1.2 of Slony-I has numerous improvements over previous
versions.
Many of them should represent near-invisible enhancements that improve
how Slony-I manages replication configuration. These include:
- A major revision of memory management to limit memory usage by slon
daemons. In earlier versions, slon would try to load 100 tuples
into memory at a time. If you have tables with Very Large bytea or
character varying columns, this could lead to loading 100 x 50MB
into memory, twice, doing terrible things to memory consumption.
slon now loads tuples in, directly, only if they are less than a
certain size, and handles those 50MB tuples individually.
- Log switching: periodically, Slony-I will switch between storing
replication data in sl_log_1 and sl_log_2, which allows regularly
cleaning these tables out via TRUNCATE and which prevents some data
loss problems relating to cases where different sets originate on
different nodes, when transaction IDs roll over the 32 bit limits.
- pg_listener is now used dramatically less than it used to be, which
diminishes the number of dead tuples you will find in this table.
This, along with log switching, should improve Slony-I behaviour on
systems where long-running transactions are common. Older versions
of Slony-I could suffer quite badly on systems that get hit by
long-running transactions.
- DDL scripts are broken into individual statements
This is more a bug fix than an enhancement; it now permits DDL
scripts to create new tables and columns, and reference them later
in the script.
In the past, DDL was submitted to the postmaster as a single query,
which meant that all of them had to reference the state of
pg_catalog as it was before the DDL ran. So you could add as many
columns to tables as you liked; you could NOT, then, reference those
columns, because the query processor would discover that the new
column didn't exist as at "before the DDL ran."
There is now a statement parser which splits scripts into individual
SQL statements and passes them to the database back end
individually.
- Slony-I tables are now marked "WITHOUT OIDS" so that they do not
consume OIDS.
It's only particularly important for sl_log_1/sl_log_2/sl_seqlog,
but the change has been applied to all the tables Slony-I uses.
UPGRADE FUNCTIONS will remove OIDs from Slony-I tables in existing
schemas, too.
- When possible (based on log switching functionality), partial
indexes on sl_log_1 and sl_log_2 are created on a per-origin-node
basis. This provides the performance boost of having an easily
recognisable index, but without the risk of having XIDs from
different nodes mixed together in one index, where roll-over could
Cause Problems...
These features are generally configurable, but the defaults ought to
allow improved behaviour for all but the most "Extreme Uses."
There are also numerous enhancements that are more directly visible:
- Windows support
A group of developers has contributed changes to allow running
Slony-I as a Windows service.
- PostgreSQL Version Compatibility Changes
Slony-I developers have been following the 8.2 release expected in
fall 2006; Slony-I 1.2 can be expected to work with it without
problems.
Compatibility with version 7.3 has been dropped; version 1.2 notably
makes use of some array functionality that didn't exist in 7.3. If
you are on 7.3, you're on a REALLY OLD version of PostgreSQL, and
really ought to upgrade. You may use Slony-I 1.1.5 to get to a more
modern version, and then upgrade to 1.2 as needed.
- Process structure is revised considerably; in 1.1.0, an attempt was
made to create an internal 'watchdog' that would restart threads
that fell over for whatever reason.
In 1.2, this is massively revised. The "main" slon thread should,
as of 1.2, never fall down, which means that there should no longer
be a need for watchdog processes to watch carefully.
This makes slon very much less fragile than it was in earlier
versions. This may revise how you want to start it up... In the
past, slon processes tended to fall over easily, mandating having
some form of "watchdog." The new behaviour points more towards
"init" / "rc.d"-like handling, where, upon boot-up, one "rc.d" script
might start up PostgreSQL, another one starts pgpool, and a third
(which must be invoked after PostgreSQL is started) starts up a
slon.
- Subscribe set aggressively locks tables on the subscriber to avoid
failures
This may become a configurable option so that sites where they
Really Know What They're Doing and are sure that they won't deadlock
themselves can leave the tables more open.
But for most users, it's really preferable to lock the tables down
on the subscriber so that you don't get 18 hours into subscribing an
18GB replication set, hit a deadlock, and have to start all over.
(The slon will automatically try again; the irritation is that you
may have been depending on that getting done by Monday morning...)
- As has been the case for fairly much each release that has been
made, the documentation has been significantly extended. The "admin
guide" has been augmented and cleaned up.
Notable additions include a listing of "Best Practices" (due in
great part to discoveries by the oft-unsung heroes of Afilias' Data
Services department) and a fairly comprehensive listing of log
messages you may expect to see in your Slony-I logs.
- A lot of fixes to the build environment (this needs to be tested on
lots of platforms)
One noticeable upshot of this should be that (on Unix-like systems;
Windows(tm) appears to differ) you oughtn't need to specify a whole
lot of --with-pg[options] on the ./configure command line. You should
be able to get most of what's needed simply by specifying the path to
pg_config via --with-pgconfigdir.
- slon "lag interval" option
You can tell a node to lag behind by a particular interval of time.
slon -l "4 hours" [and probably other options :-)]
will cause the slon to ignore events until they reach the age
corresponding to the requested interval. That way, you can hold a
node behind by 4 hours.
This is a potential "foot gun" as there are some cases (MOVE SET,
FAILOVER) where events have to be coordinated across all nodes with
near-simultaneity.
- slon "stop after event" option
You can tell a slon to terminate itself as soon as it receives a
certain event from a certain node.
Just as with "lag interval," this could be a bit of a foot gun...
- slon "run program after each log shipped SYNC" option
You can specify a program to run each time a SYNC event
is successfully closed off.
- Bug 1538 - if there is only one node, sl_event, sl_log_1 never get
cleared out
Logic added to cleanupevent() to clear out old sl_event entries if
there is just one node. That then allows the cleanup thread to
clear sl_log_1 etc.
- Bug 1566 - Force all replication to occur in the ISO datestyle.
This ensures that we can apply date/timestamps regardless of the
datestyle they were entered in.
- Force all replication to occur with standards_conforming_strings set
to off. This ensures we can replicate a node running on 8.2 without
extra escape chars showing up in the data.