[dev.icinga.com #1110] dump configfilevariables and more multiline mbufs as bulk inserts #509
This issue has been migrated from Redmine: https://dev.icinga.com/issues/1110
Created by mfriedrich on 2011-01-12 15:53:26 +00:00
dumping the configfilevariables (from icinga.cfg and resource.cfg)
this is been invoked in idomod.c
where idomod_write_config_files does that.
this will be fetched into ido2db, where data is read from teh socket and stored into mbuf as buffer.
and the handling stuff after finished the read triggered on ENDDATA ...
the first call onto ido2db_query_insert_or_update_configfilevariables_add makes it questionable qhy an update must take place if updating everything which is within the where clause/the unique constraint.
this would be a if not existing then insert it, otherwise return the primary id (or sequence.curval).
maybe a call for a procedure doing exactly that.
but that's not the main point in here.
multiple insert combo
after having gotten the configfile_id the for loop over mbuf takes each line and does an insert into the table. so by means, single query for single line.
this should be written asl bulk query with normalized sql:
building the values string together during the loop, and sending the query afterwards.
the logic can be build upon issue #1108
and is considered to be applied for more than this example, but all possible multiline buffers.
2011-07-22 15:31:22 +00:00 by mfriedrich be5d66e
2011-07-22 17:54:44 +00:00 by mfriedrich 456cd77
2011-07-22 18:19:00 +00:00 by mfriedrich 4bd8fc1
2011-07-22 18:35:26 +00:00 by mfriedrich 26518be
2011-07-23 21:45:27 +00:00 by mfriedrich 621bb94
2011-07-23 21:51:13 +00:00 by mfriedrich 263f6d8
2011-07-23 22:01:21 +00:00 by mfriedrich cfbbbb9
2011-07-23 22:33:49 +00:00 by mfriedrich 7a17a66
2011-07-24 18:26:59 +00:00 by Tommi 9a8f61c
2011-07-31 15:00:39 +00:00 by Tommi 42902613b36654d45b9cc0d892110bdfcc75edac
2011-08-01 19:12:28 +00:00 by Tommi 791c141a210e84340bdff3972f80a50a7aaef614
2011-08-02 18:13:38 +00:00 by Tommi fd2266e
2011-08-02 18:14:27 +00:00 by Tommi 91b9d8b
2011-08-03 09:10:10 +00:00 by Tommi b1fc92b1d42ab39a00abc0d583f56413afbfad0f
2011-08-03 09:52:19 +00:00 by Tommi 145ce4f6e933e7a9a2c757d49c67438f825420ce
2011-08-03 10:38:15 +00:00 by Tommi e9b668a
2011-08-03 10:38:32 +00:00 by Tommi 786ead8
Updated by mfriedrich on 2011-07-22 15:30:19 +00:00
Updated by mfriedrich on 2011-07-22 15:57:08 +00:00
Updated by Tommi on 2011-07-24 17:01:33 +00:00
for oracle its better to use real bulk inserts with arrays(OCI_BindArray* functions) on prepared statements instead of building a long sql with literals which will causing higher parsing load and poisoning the SGA.
Updated by mfriedrich on 2011-07-24 17:53:50 +00:00
well i was not aware of that solution, either way, the combined union will increase performance with 1 query compared to 2000 queries e.g. for contactgroupmembers. if you prefer to have that binded instead, feel free to change it, it's out of my scope currently as i am working on pgsql mainly.
Updated by Tommi on 2011-07-31 15:28:56 +00:00
oracle bind array code added to my testido branch, replacing previous version. Because of partially changed logic, also libdbi code may affected, but shouldnt. It worked in my small environment but should be tested in bigger installations (5000+ services)