AWS plugin stopped processing messages #44
Comments
…onParser The Jackson ObjectMapper used in CloudtrailSNSNotificationParser was configured to fail on unknown properties (default) and thus parsing the SNS notifications failed if the format was changed in AWS. Fixes #44
Any ETA to merge and release? |
@akrus This will be released alongside Graylog 2.4 in which we start bundling this plugin by default. Currently we are working towards a first public beta release for 2.4 which will come as soon as all features are merged. |
…onParser (#47) * Prevent failing on unknown JSON properties in CloudtrailSNSNotificationParser The Jackson ObjectMapper used in CloudtrailSNSNotificationParser was configured to fail on unknown properties (default) and thus parsing the SNS notifications failed if the format was changed in AWS. Fixes #44 * Create lenient object mapper once and inject it where required
…onParser (#47) * Prevent failing on unknown JSON properties in CloudtrailSNSNotificationParser The Jackson ObjectMapper used in CloudtrailSNSNotificationParser was configured to fail on unknown properties (default) and thus parsing the SNS notifications failed if the format was changed in AWS. Fixes #44 * Create lenient object mapper once and inject it where required (cherry picked from commit 92d7cb3)
I'm afraid the problem is still not fixed:
|
@bernd @akrus While looking similar, this is a completely different problem. We'll have to check what the canonical response format for this kind of payload is. |
As documented here: http://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-cloudtrail-to-send-notifications.html it should be always a string array @akrus are these messages coming from cloudtrail notifications or is the same topic receiving notifications from other services? Seems the 2 notifications in this issue are related to AWS Config service http://docs.aws.amazon.com/es_es/config/latest/developerguide/how-does-config-work.html |
Yes, all the messages are coming from Cloudtrail. We have RDS logs and AWS config writing there (and added SES recently, but this error appeared before SES started to send logs). |
@joschi, should I try contacting Amazon on this case? Or is it possible to have a workaround for this? |
@akrus I'm checking this against the official SDK now. sorry for the inconvenience! |
@akrus Can you confirm that you are sending AWS Config via cloudtrail as described here: http://docs.aws.amazon.com/config/latest/developerguide/log-api-calls.html ? It looks like the SNS notification generated in that case is different to other cloudtrail producers. If that's so, that does look like a bug with the service itself to me. The AWS SDKs I could find all assume (wrongly) that the |
Another update: After spending quality time in the debugger, the AWS cloudtrail SDK does indeed expect and parse only Which means that those notifications @akrus has in the cloudtrail queue are in fact not cloudtrail logs, but AWS Config events, which have a different format. Long story short: The cloudtrail plugin is correct, the docs are correct, the SDK code is misleading, but correct, it is simply that the AWS Config docs are confusing. To request support for native AWS Config logs, please open a new feature request ticket. Thanks, |
Hello!
Plugin just stopped working, I can see the following in the logs:
Configuration looks fine, seems it just cannot parse the message.
The text was updated successfully, but these errors were encountered: