forked from pgq/skytools-legacy
-
Notifications
You must be signed in to change notification settings - Fork 0
/
NEWS
243 lines (174 loc) · 9.17 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
2012-11-02 - SkyTools 3.1.2 - "Tank in a Tarball"
= Features =
* --find-copy-node switch to 'add-table' that instructs 'copy'
to walk upwards until it finds node that has full table.
Parts of the patch were in earlier, but they did not work
on merge node properly. Now the walk code was moved to copy,
instead being done during add-table to fix that.
Compare and repair does not support (yet) such walking.
= Minor Features =
* Allow overrided options with --set to show up in --ini output
* Londiste: try to show exact event info on failure.
= Fixes =
* walmgr: Move .walshipping.last update AFTER rsync. Otherwise
rsync failure will not be noticed during "master sync".
* londiste execute: when ddl was already applied, connection was
left in wrong mode. Fix.
* Fix compare expression - md5() was in wrong place.
* Fix Londiste compare and repair to accept copy-condition from handler (Asko Oja)
* londiste: don't filter EXECUTE or TRUNCATE events on merge node
* qadmin: make "install pgq" also install pgq_coop
* Fix 2.1 to 3.0 upgade script, rename to pgq.upgrade_2.1_to_3.0.sql
* Reorg of Londiste tests (Asko Oja)
* Socket library autoconf (Tony Arkles)
* pgq: quote internal table names
* Bump pgq schema version to 3.1.2
2012-10-09 - SkyTools 3.1.1 - "Service Oriented Diapers"
Queue_mover and queue_splitter now have service names ending with 3.
Config files need to change: [queue_mover3], [queue_splitter3].
= Minor Features =
* scriptmgr: option for specifying service type (-t / --type)
* scriptmgr: status command can optionally be more selective
* skytools_upgrade: "dry run" option (--not-really)
* pgq.Consumer: log idle time, keepalive_stats tper
* pgq.Consumer: keepalive_stats to force logging even when idle
= Cleanups =
* qadmin.py: "show version" command
* qadmin.py: unified results displaying
* skytools.scripting: added shutdown() to BaseScript
* skytools.scripting: log config reload, updated comment.
* skytools.__init__: added symbols for natsort module
* qadmin.py: updated script version, added pgq version, amended messages
* scripts: changed queue_mover and queue_splitter service_name to end with '3'
= Fixes =
* pgqd: make connections handling more robust
* londiste compare/repair: when provider is branch, stop it's worker to get sync
* walmgr: Don't complain if a nonexistent WAL file is encountered during cleanup.
* pgq_node.demote_root: decrease lock level
* londiste.local_remove_table: resture DDL if still attached to table
* pgq_coop.next_batch: check and create subconsumers after locking main consumer
* --sync-watermark: don't allow subtree wm get above upstream wm
* londiste compare: calculate common column names
* londiste compare: use md5() if comparing 8.3 with 8.4+
* Use python interpreter given by '--with-python=PYTHON' everywhere
2012-07-19 - SkyTools 3.1 - "Carnival-on-a-Stick"
= Features =
* londiste: Wait commands
> londiste $ini add-table .. --wait-sync
> londiste $ini wait-sync
Wait until all tables are fully copied over.
> londiste $ini wait-root
Wait for next tick on root reaches local node.
> londiste $ini wait-provider
Wait for next tick on provider reaches local node.
* londiste execute: support meta attributes in sql
This allows execute ADD COLUMN from root, over whole
cascade even when not all nodes contain the table.
--*-- Local-Table: mytable
ALTER TABLE mytable ADD COLUMN foo text;
* Convert DB modules to extensions (9.1+). Now following modules
are available as extensions: pgq, pgq_node, pgq_coop, pgq_ext, londiste.
All the old files are kept in old locations to avoid any breakage.
Known problems in Postgres extensions infrastructure (as of 9.1.4):
- pg_dump crashes when extensions has sequence whose value needs to be dumped.
Thus pgq.batch_id_seq cannot be made dumpable, thus it's value
gets lost during dump+reload.
- CREATE EXTENSION pgq_coop FROM unpackaged; + DROP EXTENSION pgq_coop;
will not drop schema. Plain CREATE+DROP works fine.
= Minor Features =
* londiste.applied_execute: drop queue_name from pkey
* pgq.LocalConsumer: consumer that tracks processed ticks in local file
* simple_local_consumer: LocalConsumer that runs query for each event.
* pgq.Consumer: 'table_filter' config param, filters on ev_extra1,
where PgQ triggers put source table name.
* londiste.periodic_maintenance: cleanup for applied_execute
* pgq.maint_operations: include londiste.periodic_maintenance
* skytools.exists_view(): new function
* skytools.fileutil: new module, contains write_atomic()
* londiste.handler: make encoding= param available to part & londiste handlers
* debian: build modules for all installed server-dev versions
= Fixes =
* CascadedConsumer: re-initialize provider connection when location changes
* pgq_node.drop_node: mention node name in info message
* drop-node: move find_root before local drop
* pgq.maint_rotate_tables: check txid sanity, fail if bad
* sql_tokenizer: allow idents starting with underscore
* BaseScript: write pidfile atomically to avoid corrupt pidfiles.
* londiste replay: skip provider checks if copy_node is used
* CascadedWorker: don't move watermark on source further than incoming batch.
2012-05-30 - SkyTools 3.0.3 - "Biometric Authentication by Yodeling"
= Features =
* londiste copy: copy table from another node (add-table --copy-node=NODE)
* londiste remove-table: drop sequence too
* public.create_partition: move under londiste schema, it's now generic enough
* londiste.create_partitions: Support copying rules from master table
* handlers.dispatch: use londiste.create_partitions, use old func as fallback
* walmgr: add option for init-slave to add password from file to .pgpass
* walmgr: add command synch-standby
= Fixes =
* CascadedWorker: merge-leaf-to-branch needs to publish wm info
* pgq_node.create_node: create branch nodes with disable_insert set
* pgq.insert_event: ignore disable_insert if in 'replica' role
* create_partition: public grant needs special casing
* skytools.dbstruct: better rule name redirection
* debian: add build-deps, force debian/control rebuild
* pgq_node.unregister_location: do automatic unsubscribe
* pgq_node.drop_node: drop queue data even if node_info is empty
* londiste add-table: --expect-sync does not require table existance on provider
= Cleanups =
* skytools_upgrade: show old version before upgrade
* CascadeAdmin: add node name prefix messages from db
* handlers.dispatch: add comment about differences from part_func
* londiste.find_table_oid: drop old version - 9.1+ panics when argument name changes
* dbservice.get_record_list: do not use parse_pgarray on python lists
= Win32 =
* skytools.scripting: catch EINTR from sleep()
* signal_pidfile: support sig=0 on win32
* skytools.scripting: detect if signal name exists
= Schema version changes =
* pgq_node.version(): 3.0.0.18
* londiste.version(): 3.0.0.16
2012-05-10 - SkyTools 3.0.2 - "Consumes 30% Less Brain Than Competing Products"
= Features =
* Draft implementation of fine-grained grants for SQL
functions. See commit doc/sql-grants.txt for details.
New roles are created during upgrade, but new grants
need to be applied manually.
* pgq.create_queue: instead pre-determined grants,
copy grants for event tables from pgq.event_template.
* simple_consumer.py script, for simply event processing
by simply launching a SQL function for each event.
* londiste.handlers.part: make hash function configurable
* psycopgwrapper: allow name= argument to .cursor()
= Fixes =
* londiste: Always use TRUNCATE CASCADE
* pgq.cascade.CascadedWorker: publish_local_wm() is missing a commit
* walmgr: fix backup from slave (Sergey Burladyan)
= Cleanups =
* pgq.insert_event(): outer wrapper does not need to be secdef
* source.cfg: minor improvements
* sql/pgq: use terse verbosity in regtests
2012-04-18 - SkyTools 3.0.1 - "All The Snow You Can Eat"
= Features =
* skytools.config: new .getbytes() method to support
human-readable sizes.
= Fixes =
* skytools.skylog: Wrap logger.handlers.SysLogHandler to work
around broken BOM addition when logging unicode() strings.
* skytools.skylog: Improve compatibility with direct
logger module usage.
* debian/skytools.ini: include in tgz
* pgqd(libusual): ignore EINTR on close()
* Better error message in londiste.handler.dispatch
when part_field is NULL.
* Bump internal version in pgq/pgq_coop/pgq_node schemas.
Seems I forgot to do it for 3.0, probably because
there were doc-only changes in SQL code.
2012-03-17 - SkyTools 3.0 - "Business Advantage"
= Major New Features =
* Cascaded queues
* Londiste: Parallel copy
* Londiste: EXECUTE
* Londiste: handlers
* QAdmin
* pgqd