Skip to content
Newer
Older
100644 433 lines (293 sloc) 16.7 KB
64a4275 @tardate transfered sources collection to github
authored
1 # ============================================================
2 # Notes on the OCCI MQ Demo
3 #
4 # See http://tardate.blogspot.com/2007/06/mq-and-occi-demo.html
5 # Prepared by Paul Gallagher <gallagher.paul@gmail.com>
6 # $Id: occimqdemo-readme.txt,v 1.5 2007/06/09 02:38:02 paulg Exp $
7 # ============================================================
8
9 What this sample demonstrates:
10 1. C++ (OCCI) Oracle database access
11 2. Transparent Application Failover (TAF) notifications in C++ (OCCI)
12 3. Building a C++ application with MQ and OCCI support
13
14 What this sample does NOT demonstrate:
15 1. MQ + DB operations within a single distributed transaction
16
17 Platform:
18 1. Code is written to run on Linux x86. See the make file for a hint on how this can be easily adapted to 64bit Linux (e.g. ZSeries Linux 64)
19 2. Requires Websphere MQ 6.0 for Linux (free download from IBM)
20 3. Requires Oracle Database (free download from Oracle). Works with either full client or Instant Client.
21
22
23 For More Information
24 ==============================================================================
25 See http://tardate.blogspot.com/2007/06/mq-and-occi-demo.html for more
26 description of the demo and access to the latest demo download.
27
28 For information on programming OCCI, see the OCCI home page on OTN
29 http://www.oracle.com/technology/tech/oci/occi/index.html
30
31 For more information about Oracle Database, see
32 http://www.oracle.com/technology/documentation/database10gr2.html
33
34
35 Overview of the Basic Flow to run this Demo
36 ==============================================================================
37 The following sections are included in this file.
38
39 1. Check Current Limitations/Untested Aspects
40
41 2. Obtain and Install Oracle Database 10g
42
43 3. Obtain and Install Websphere MQ
44
45 4. setup the mqm user for Oracle
46
47 5. Setting up ORACLE tnsnames.ora
48
49 6. Setting up MQ OCCIMQDEMO environment
50
51 7. Build the OCCIMQDEMO sample programs
52 7a. Initialising ORACLE OCCIMQDEMO schema for the demo
53 7b. Build the C++ sample programs
54 7c. Verifying Correct Build Libraries for OCCI
55
56 8. Running the demo
57
58 9. Generate a Test Report
59
60 10. Checking Queue Status
61
62 Assuming Oracle Database and MQ already installed, the commands for a full
63 build/test cycle are as follows:
64
65 ./qcontrol.sh initqm
66 ./qcontrol.sh createq 1..15
67 ./build.sh initdb
68 ./build.sh db
69 ./loadTest.sh run 1..15
70 ./loadTest.sh report 1..15
71 ./qcontrol.sh delqm
72 ./build.sh dropdb
73
74
75
76 1. Current Limitations/Untested Aspects
77 ==============================================================================
78 1. MQ and Database operations are not conducted in a distributed transaction.
79 This means that an MQ dequeue will still succeed if, for example, a database write fails.
80 The sample programs correctly report this behaviour if it should occur.
81
82 2. MQ failover not tested.
83 Additional exception handling may be required.
84
85 3. Database failover to DR not tested.
86
87 4. If queues are not empty at the start of a test, mqproducer and mqconsumer
88 iterations may not tally correctly. Use (./qcontrol.sh clear x..y) to clear queues.
89
90 5. Currently, Oracle only "officially" support GCC 3.2.3 with 10g Release 2 (10.2) for IBM zSeries Based Linux
91 (see Oracle® Database Release Notes http://download-west.oracle.com/docs/cd/B19306_01/relnotes.102/b25399/toc.htm)
92 In practice, I have been able to get this demo to work with GCC 3.4.3 by soft linking the required .so file names
93
94
95
96 2. Obtain and Install Oracle Database 10g
97 ==============================================================================
98 This sample requires an Oracle Database server. Only 10g has been tested, but there
99 is nothing in the sample that shouldn't work with other versions of the server.
100
101 If you do not already have an Oracle Database server available, see Oracle OTN
102 to obtain free developer versions:
103 http://www.oracle.com/technology/software/products/database/oracle10g/index.html
104
105 Follow the Oracle installation instructions to setup and test the database server.
106
107
108
109 3. Obtain and Install Websphere MQ
110 ==============================================================================
111 This sample requires a Websphere MQ server. Only Websphere MQ 6.0 for Linux has been tested.
112
113 A trial version of Websphere MQ is available for download from IBM (http://www.ibm.com)
114
115 See Websphere documentation and information available at
116 http://www-306.ibm.com/software/integration/wmq/library/?S_CMP=rnav
117
118
119
120 4. Setting up mqm user for ORACLE environment
121 ==============================================================================
122
123 Recommended approach is to install Oracle Instant Client for the mqm user.
124 This means mqm user does not need any special oracle installation privileges.
125
126 A script (installInstantClient.sh) has been provided to install and config the Instant Client.
127 Installation procedure is as follows:
128
129 1. Change to the directory where the instant client directory should be held. mqm home is fine:
130 [mqm@tintin occimqdemo]$ cd
131
132 2. Copy the 3 Instant Client zip files into this directory
133 [mqm@tintin mqm]$ cp /downloads/instantclient-basic-linux32-10.2.0.3-20061115.zip .
134 [mqm@tintin mqm]$ cp /downloads/instantclient-sdk-linux32-10.2.0.3-20061115.zip .
135 [mqm@tintin mqm]$ cp /downloads/instantclient-sqlplus-linux32-10.2.0.3-20061115.zip .
136
137 3. Run the installInstantClient.sh script
138 [mqm@tintin mqm]$ occimqdemo/installInstantClient.sh
139
140 this will unpack and fixup the installation ... read the output
141
142 4. Fixup the .bash_profile file as instructed.
143 e.g. you will be told to add the following lines:
144 export ORACLE_HOME=/var/mqm/instantclient_10_2
145 export LD_LIBRARY_PATH=$ORACLE_HOME/lib
146 export PATH=$PATH:$ORACLE_HOME/bin
147 export ORACLE_SID=OCCIMQDEMO
148
149 5. Configure tnsnames.ora for your database
150 see next section (Setting up ORACLE tnsnames.ora)
151
152 6. Re-login to mqm user, should be ready to go
153
154
155
156 5. Setting up ORACLE tnsnames.ora
157 ==============================================================================
158
159 "OCCIMQDEMO" is the standard database connection name used in the scripts. Set this up
160 appropriately in the $ORACLE_HOME/network/admin/tnsnames.ora file.
161
162 For example, a single-instance database connection:
163 OCCIMQDEMO =
164 (DESCRIPTION=
165 (ADDRESS=(PROTOCOL=TCP)(HOST=tintin.urion.com)(PORT=1521))
166 (CONNECT_DATA=
167 (SERVER=DEDICATED)
168 (SERVICE_NAME=ORCL.tintin.urion.com)
169 )
170 )
171
172 For example, a RAC connection:
173 OCCIMQDEMO =
174 (DESCRIPTION=
175 (ADDRESS_LIST =
176 (ADDRESS=(PROTOCOL=TCP)(HOST=tintin-rac1vip.urion.com)(PORT=1521))
177 (ADDRESS=(PROTOCOL=TCP)(HOST=tintin-rac2vip.urion.com)(PORT=1521))
178 (FAILOVER=yes)
179 (LOAD_BALANCE=yes)
180 )
181 (CONNECT_DATA=
182 (SERVER=DEDICATED)
183 (SERVICE_NAME=MYRAC.tintin.urion.com)
184 (FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC)(RETRIES=1)(DELAY=5))
185 )
186 )
187
188
189
190
191 6. Setting up MQ OCCIMQDEMO environment
192 ==============================================================================
193 MQ Environment setup tasks are built into the qcontol.sh script. The two
194 required setup steps are to create a queue manager (initqm), and create
195 some queues (createq)
196
197 [mqm@tintin occimqdemo]$ ./qcontrol.sh
198
199 OCCI/MQ Demo MQ Control Script
200
201 Usage:
202 ./qcontrol.sh help ... this message
203 ./qcontrol.sh initqm ... create and start the queue manager
204 ./qcontrol.sh startqm ... start the queue manager
205 ./qcontrol.sh stopqm ... stop the queue manager
206 ./qcontrol.sh delqm ... stop and delete the queue manager
207 ./qcontrol.sh createq qRange ... create queue/queue range
208 ./qcontrol.sh qstatus qRange ... show status of queue/queue range
209 ./qcontrol.sh clearq qRange ... clear queue/queue range
210 ./qcontrol.sh deleteq qRange ... delete queue/queue range
211
212 Where
213 qRange = queue number or range {a | a..b}
214 (queue number "a", or queues from "a" to "b")
215
216
217 Setup example:
218 1. create and start the OCCIMQ.QMgr queue manager
219 [mqm@tintin occimqdemo]$ ./qcontrol.sh initqm
220
221 2. create 100 queues:
222 [mqm@tintin occimqdemo]$ ./qcontrol.sh createq 1..100
223
224
225
226
227 7. Build the OCCIMQDEMO sample programs
228 ==============================================================================
229 Building the sample program (C++ and database schema) is controlled with
230 the build.sh script.
231
232
233 [mqm@tintin occimqdemo]$ ./build.sh
234
235 Demo build script for OCCI/MQ Demo dbLibrary.
236
237 Usage - OCCIMQDEMO Schema Database operations:
238 ./build.sh initdb ... initialise the demo database (create the
239 occimqdemo user and sample tables)
240 ./build.sh testdb ... simple PL/SQL test of the occimqdemo user
241 and sample tables
242 ./build.sh cleandb ... cleanout all the occimqdemo tables
243 ./build.sh dropdb ... to drop all occimqdemo objects
244 ./build.sh dbstatus ... show db status
245
246 Usage - C++ sample code operations. Generates:
247
248 ./build.sh stub [verbose] ... build the stub sources, optionally with verbose
249 output on if specified
250 ./build.sh db [verbose] ... build the db-integrated sources, optionally with
251 verbose output on if specified
252 ./build.sh demodb ... build the dbLibrary demo using Oracle demo_rdbms.mk
253 (Use this to get platform-specific library info
254 that may need to be updated in occimq.mk)
255 ./build.sh clean ... clean up compiled files
256
257
258
259 7a. Initialising ORACLE OCCIMQDEMO schema for the demo
260 ------------------------------------------------------------------------------
261 Schema management is built into the build.sh script.
262
263 Example usage:
264
265 1. create the occimqdemo user and sample tables (will prompt for system password):
266 [mqm@tintin occimqdemo]$ ./build.sh initdb
267
268 2. cleanup the schema after test runs (resets the schema - will delete all information)
269 [mqm@tintin occimqdemo]$ ./build.sh cleandb
270
271 3. completely drop the schema:
272 [mqm@tintin occimqdemo]$ ./build.sh dropdb
273
274
275
276 7b. Build the C++ sample programs
277 ------------------------------------------------------------------------------
278 C++ sample programs include:
279 bin/mqproducer (executable test program)
280 bin/mqconsumer (executable test program)
281 bin/dblibrary_test (executable db test program)
282
283 Examples for building the C++ test program:
284
285 1. Standard build - with database support, verbose output disabled:
286 [mqm@tintin occimqdemo]$ ./build.sh db
287
288 2. Standard build - with database support, verbose output enabled:
289 [mqm@tintin occimqdemo]$ ./build.sh db verbose
290
291 3. Stubbed build - programs without "dummy" db support (no Oracle environment required)
292 [mqm@tintin occimqdemo]$ ./build.sh stub verbose
293
294
295
296 7c. Verifying Correct Build Libraries for OCCI
297 ------------------------------------------------------------------------------
298 The makefile (occimq.mk) is configured to build with OCCI libraries
299 appropriate for Oracle 10gR2 on Linux x86.
300
301 To build on other environments, the libraries and includes may need to be
302 adjusted. Specifically, the CFLAGS, ORACFLAGS, LFLAGS and ORALFLAGS variables
303 in occimq.mk may need to be revised.
304
305 The easiest way to do this is to build the dblibrary_test program with the
306 makefile distributed with oracle, examine the libraries used, and update
307 occimq.mk accordingly.
308
309 The build.sh script has an operation to do such a build:
310
311 ./build.sh demodb
312
313 This will work provided the platform you are using contains the
314 $ORACLE_HOME/rdbms/demo/demo_rdbms.mk file. If not, consult the database
315 documentation for the platform concerned.
316
317
318
319 8. Running the demo
320 ==============================================================================
321 Program execution is built into the loadtest.sh script:
322
323 [mqm@tintin occimqdemo]$ ./loadTest.sh
324
325 OCCI/MQ Demo Test Runner Script
326
327 Usage:
328 ./loadTest.sh help ... this message
329 ./loadTest.sh run qRange pTimeout pThinktime cTimeout
330 ... runs test pairs (producer/consumer) for specified queues
331 ./loadTest.sh prod qRange pTimeout pThinktime
332 ... only starts a producer for specified queues
333 ./loadTest.sh cons qRange cTimeout
334 ... only starts a consumer for specified queues
335 ./loadTest.sh tail qRange ... tails log files for test programs for specified queues
336 ./loadTest.sh report qRange ... generates report from logfiles for specified queues
337
338
339 Where
340 qRange = queue number or range {a | a..b}
341 (queue number "a", or queues from "a" to "b")
342 pTimeout = producer timeout in seconds (default=120)
343 pThinktime = producer message injection delay (default=0)
344 cTimeout = consumer timeout in seconds (default=30)
345
346
347
348 NB: running the "load" operation will implicitly call "tail" after the programs have all been started.
349
350 Example usage:
351
352 1. Run a test with 1 producer-consumer pair for 20 seconds:
353 [mqm@tintin occimqdemo]$ ./loadTest.sh run 1 20
354
355 2. Run a test with 15 producer-consumer pairs for 120 seconds:
356 [mqm@tintin occimqdemo]$ ./loadTest.sh run 1..15 120
357
358 3. Watch the logs in real-time during a load test for queues 5 to 10:
359 NB: running the "run" operation will implicitly call "tail" after the programs have all been started.
360 [mqm@tintin occimqdemo]$ ./loadTest.sh tail 5..10
361
362 4. Generate a report from a run for queues 1 to 10
363 [mqm@tintin occimqdemo]$ ./loadTest.sh report 1..10
364
365
366
367 9. Generate a Test Report
368 ==============================================================================
369 Report generation is built into the loadtest.sh script:
370
371 1. Generate a report from a run for queues 1 to 10
372 [mqm@tintin occimqdemo]$ ./loadTest.sh report 1..10
373
374 This will procude a report which could be redirected to a text file and analysed with Excel.
375 Example output (admittedly on a slow machine):
376
377 Instance, Producer Iterations, Producer Msg Sent OK, Producer Msg Reply OK, Consumer Iterations, Consumer Msg In OK, Consumer Msg Saved to DB OK, Consumer Msg Out OK, Test Duration(secs), Producer Iterations per second, Average Response Time, Count Response Time > 1, Slowest Response Time > 1, Producer Errors, Consumer Errors, TAF Failovers
378 1, 43235, 43235, 43235, 43235, 43235, 43235, 43235, 600, 72, 0.0103713, 18, 3.09328, 0, 0, 0
379 2, 42872, 42872, 42872, 42872, 42872, 42872, 42872, 600, 71, 0.0106544, 18, 3.09214, 0, 0, 0
380 3, 42867, 42867, 42867, 42867, 42867, 42867, 42867, 600, 71, 0.0106341, 18, 3.09163, 0, 0, 0
381 4, 43126, 43126, 43126, 43126, 43126, 43126, 43126, 600, 71, 0.0104773, 18, 3.09137, 0, 0, 0
382 5, 42970, 42970, 42970, 42970, 42970, 42970, 42970, 600, 71, 0.0105106, 18, 3.09185, 0, 0, 0
383 6, 42915, 42915, 42915, 42915, 42915, 42915, 42915, 600, 71, 0.0105912, 18, 3.09264, 0, 0, 0
384 7, 43167, 43167, 43167, 43167, 43167, 43167, 43167, 600, 71, 0.0105066, 18, 3.092, 0, 0, 0
385 8, 43249, 43249, 43249, 43249, 43249, 43249, 43249, 600, 72, 0.010494, 18, 3.09296, 0, 0, 0
386 9, 43028, 43028, 43028, 43028, 43028, 43028, 43028, 600, 71, 0.010564, 18, 3.09189, 0, 0, 0
387 10, 43218, 43218, 43218, 43218, 43218, 43218, 43218, 600, 72, 0.0104776, 18, 3.093, 0, 0, 0
388
389
390
391 Notes on the above report:
392 1) producer and consumer itermations and successful msg send/receieve all tally, which is what we want (no dropped messages)
393 2) 15 messages fell outside the 1sec threashold for a reply, which is a concern. This was a slow/overloaded machine so perhaps expected.
394 3) to compare results without the database access, you could re-run the test after building with the dbstub:
395 ./build.sh stub
396 ./loadTest.sh run 1..10 600
397 ./loadTest.sh report 1..10
398
399 For example, this now reports:
400 Instance, Producer Iterations, Producer Msg Sent OK, Producer Msg Reply OK, Consumer Iterations, Consumer Msg In OK, Consumer Msg Saved to DB OK, Consumer Msg Out OK, Test Duration(secs), Producer Iterations per second, Average Response Time, Count Response Time > 1, Slowest Response Time > 1, Producer Errors, Consumer Errors, TAF Failovers
401 1, 57529, 57529, 57529, 57529, 57529, 57529, 57529, 600, 95, 0.00220925, 1, 1.28549, 0, 0, 0
402 2, 57366, 57366, 57366, 57366, 57366, 57366, 57366, 600, 95, 0.00211662, 1, 1.31558, 0, 0, 0
403 3, 57590, 57590, 57590, 57590, 57590, 57590, 57590, 600, 95, 0.00213661, 1, 1.28589, 0, 0, 0
404 4, 57771, 57771, 57771, 57771, 57771, 57771, 57771, 600, 96, 0.00212538, 1, 1.30506, 0, 0, 0
405 5, 57591, 57591, 57591, 57591, 57591, 57591, 57591, 600, 95, 0.00216829, 1, 1.30558, 0, 0, 0
406 6, 57655, 57655, 57655, 57655, 57655, 57655, 57655, 600, 96, 0.00208699, 1, 1.30506, 0, 0, 0
407 7, 57675, 57675, 57675, 57675, 57675, 57675, 57675, 600, 96, 0.00212047, 1, 1.305, 0, 0, 0
408 8, 57704, 57704, 57704, 57704, 57704, 57704, 57704, 600, 96, 0.00205405, 1, 1.3043, 0, 0, 0
409 9, 57578, 57578, 57578, 57578, 57578, 57578, 57578, 600, 95, 0.00217947, 1, 1.30432, 0, 0, 0
410 10, 57705, 57705, 57705, 57705, 57705, 57705, 57705, 600, 96, 0.00197397, 1, 1.29046, 0, 0, 0
411
412
413 10. Checking Queue Status
414 ==============================================================================
415 Queue control is built into the qcontol.sh script (see "Setup MQ Environment" above).
416
417 Sample usage:
418
419 1. Show the status of queues 5 to 10:
420 [mqm@tintin occimqdemo]$ ./qcontrol.sh qstatus 5..10
421
422 2. Clear queues 1 to 10:
423 [mqm@tintin occimqdemo]$ ./qcontrol.sh clearq 1..10
424
425
426
427
428
429
430
431
432
Something went wrong with that request. Please try again.