Improve logging. #104
Improve logging. #104
Conversation
…stamp that's malformed or missing. This will significantly reduce our map tasks' log size.
Remove spam logging.
This reverts commit 3b15ed7.
* Add deduplicator class. Create a non-scheduled deduplication task: it will need to be updated manually in Azkaban with the desired params.
adds dedup too, intentional? |
This reverts commit 83b10b9.
…n Camus class we want logs in Splunk for.
So you're mapping |
Ugh, thanks for catching that. And I think I will remove the debug printout. When I tested in staging I expected the debug to be there, but for the wrong reason. Will update the PR |
To be honest, this is not a pretty solution.
I've spent quite a bit of time trying to figure out how to redirect log4j, but to no result 😞 |
I figured you'd done that. Next best thing is good enough. |
event_timestamp
spammy logging.kafka.max.pull.hrs
confusing messagestdout
so Splunk picks up the logsSee staging output in Splunk
Closes https://github.com/Shopify/camus-project/issues/11 with the caveat that only some hand-picked important Camus longs will appear in Splunk.