Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #10554] Non-UTF8 characters from plugins causes IDO to fail #3613

Closed
icinga-migration opened this issue Nov 6, 2015 · 11 comments
Closed

Comments

@icinga-migration
Copy link
Member

@icinga-migration icinga-migration commented Nov 6, 2015

This issue has been migrated from Redmine: https://dev.icinga.com/issues/10554

Created by dgoetz on 2015-11-06 12:35:23 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2015-12-10 11:30:04 +00:00)
Target Version: 2.4.2
Last Update: 2016-02-23 09:58:17 +00:00 (in Redmine)

Icinga Version: 2.3.11
Backport?: Already backported
Include in Changelog: 1

A plugin giving output with non-UTF8 characters causes IDO to fail. Because we can not make sure every plugin handles this correctly. Can we get Icinga 2 to ignore such plugin output?

An example:

[2015-11-06 13:08:17 +0100] critical/IdoPgsqlConnection: Error "ERROR: 
invalid byte sequence for encoding "UTF8": 0xf6 0x73 0x74 0x65
" when executing query "UPDATE icinga_servicestatus SET 
acknowledgement_type = E'0',  active_checks_enabled = E'1',  check_command 
= E'nrpe',  check_source = E'su01k027.xxx.XXX.de', 
check_timeperiod_object_id = 79306,  check_type = E'0', 
current_check_attempt = E'1',  current_notification_number = E'2', 
current_state = E'1',  endpoint_object_id = 79031,  event_handler = E'', 
event_handler_enabled = E'1',  execution_time = E'0.62169599533081055', 
flap_detection_enabled = E'0',  has_been_checked = E'1',  instance_id = 1, 
 is_flapping = E'0',  is_reachable = E'1',  last_check = 
TO_TIMESTAMP(1446790307),  last_hard_state = E'1',  last_hard_state_change 
= TO_TIMESTAMP(1446798595),  last_notification = TO_TIMESTAMP(1446801203), 
 last_state_change = TO_TIMESTAMP(1446798595),  last_time_ok = 
TO_TIMESTAMP(1443419904),  last_time_warning = TO_TIMESTAMP(1446798595), 
latency = E'0',  long_output = E'',  max_check_attempts = E'1', 
modified_service_attributes = E'0',  next_check = 
TO_TIMESTAMP(1446876284),  next_notification = TO_TIMESTAMP(1446815603), 
normal_check_interval = E'1440',  notifications_enabled = E'1',  output = 
E'WARNING - (261 warnings) - 11/06/2015 04:45:43 ANS1228E Sending of 
object 
''/home/aea11/dat/domino/html/sht/www/images/europakarte/karte▒ster.JPG'' 
failed ...',  passive_checks_enabled = E'1',  percent_state_change = 
E'99',  perfdata = E'dsmerror_log_lines=10348 dsmerror_log_warnings=261 
dsmerror_log_criticals=0 dsmerror_log_unknowns=0', 
problem_has_been_acknowledged = E'0',  process_performance_data = E'1', 
retry_check_interval = E'1',  scheduled_downtime_depth = E'0', 
service_object_id = 94836,  should_be_scheduled = E'1',  state_type = 
E'1',  status_update_time = TO_TIMESTAMP(1446811697) WHERE 
service_object_id = 94836"
Context:
        (0) Reconnecting to PostgreSQL IDO database 'ido-pgsql'

Attachments

Changesets

2015-12-10 11:26:19 +00:00 by (unknown) fba0b49

Make sure that all strings in the IDO database are UTF8-encoded

fixes #10554

2016-02-23 08:08:32 +00:00 by (unknown) 2e93082

Make sure that all strings in the IDO database are UTF8-encoded

fixes #10554
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 18, 2015

Updated by gbeutner on 2015-11-18 08:49:35 +00:00

  • Category set to DB IDO
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 18, 2015

Updated by gbeutner on 2015-11-18 08:59:36 +00:00

The underlying problem is that there's no way to determine which encoding things like plugin output and config object names are in.

One idea would be to set the client encoding to latin1:

set client_encoding to 'latin1'
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 18, 2015

Updated by gbeutner on 2015-11-18 19:19:06 +00:00

  • Status changed from New to Feedback
  • Assigned to set to mfriedrich
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 23, 2015

Updated by mfrosch on 2015-11-23 09:04:30 +00:00

  • Description updated
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 23, 2015

Updated by mfrosch on 2015-11-23 15:56:49 +00:00

Best way seems to be setting connection encoding to 'latin1', since the IDO is modelled for latin.

Though it needs testing if we eradicate these problems with that change.

Icinga 1 would have similar problems with incorrect chars, but will write it to database, due to latin1.

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 23, 2015

Updated by mfriedrich on 2015-11-23 16:15:19 +00:00

  • Status changed from Feedback to New
  • Assigned to deleted mfriedrich
  • Priority changed from Normal to High
  • Target Version set to Backlog
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Nov 24, 2015

Updated by mfriedrich on 2015-11-24 13:04:52 +00:00

Quick workaround: Fix the plugin output and strip non-utf8 characters before passing the output back to Icinga 2.

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 10, 2015

Updated by mfriedrich on 2015-12-10 11:27:02 +00:00

  • Status changed from New to Assigned
  • Assigned to set to gbeutner
  • Target Version changed from Backlog to 2.4.2
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 10, 2015

Updated by Anonymous on 2015-12-10 11:30:04 +00:00

  • Status changed from Assigned to Resolved
  • Done % changed from 0 to 100

Applied in changeset fba0b49.

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 10, 2015

Updated by mfriedrich on 2015-12-10 11:39:43 +00:00

  • File added i2_10554_pgsql_backend.png
  • File added i2_10554_fixed.png

Fixed

i2_10554_pgsql_backend.png

i2_10554_fixed.png

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Feb 23, 2016

Updated by gbeutner on 2016-02-23 09:58:17 +00:00

  • Backport? changed from Not yet backported to Already backported
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.