Skip to content
This repository has been archived by the owner on Apr 27, 2021. It is now read-only.

Improve logging. #104

Merged
merged 12 commits into from Nov 17, 2017
Merged

Improve logging. #104

merged 12 commits into from Nov 17, 2017

Commits on Oct 27, 2017

  1. Add UTF8 flags

    @datwright
    datwright committed Oct 27, 2017
    Configuration menu
    Copy the full SHA
    3b15ed7 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2017

  1. Remove outputing a log line for individual message that has evet_time… 

    …stamp that's malformed or missing.

This will significantly reduce our map tasks' log size.
    Olessia Karpova committed Nov 6, 2017
    Configuration menu
    Copy the full SHA
    7199651 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #102 from Shopify/remove_spam_logging 

    Remove spam logging.
    Olessia Karpova committed Nov 6, 2017
    Configuration menu
    Copy the full SHA
    ac5664c View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2017

  1. Revert "Add UTF8 flags" 

    This reverts commit 3b15ed7.
    Olessia Karpova committed Nov 7, 2017
    Configuration menu
    Copy the full SHA
    46af2ed View commit details
    Browse the repository at this point in the history

  2. De-duplication flow (#101) 

    * Add deduplicator class.
Create a non-scheduled deduplication task: it will need to be updated manually in Azkaban with the desired params.
    Olessia Karpova committed Nov 7, 2017
    Configuration menu
    Copy the full SHA
    83b10b9 View commit details
    Browse the repository at this point in the history

  3. Remove more logging.

    Olessia Karpova committed Nov 7, 2017
    Configuration menu
    Copy the full SHA
    891dd63 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into staging

    Olessia Karpova committed Nov 7, 2017
    Configuration menu
    Copy the full SHA
    7cce7b0 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2017

  1. Revert "De-duplication flow (#101)" 

    This reverts commit 83b10b9.
    Olessia Karpova committed Nov 9, 2017
    Configuration menu
    Copy the full SHA
    f3cb450 View commit details
    Browse the repository at this point in the history

  2. Update the log message for kafka.max.pull.hrs; post metric to Datadog.

    Olessia Karpova committed Nov 9, 2017
    Configuration menu
    Copy the full SHA
    a437538 View commit details
    Browse the repository at this point in the history

  3. Also publish messages to stdout.

    Olessia Karpova committed Nov 9, 2017
    Configuration menu
    Copy the full SHA
    3eb66a3 View commit details
    Browse the repository at this point in the history

  4. Add a log wrapper class that also outputs to stdout. Use that class i… 

    …n Camus class we want logs in Splunk for.
    Olessia Karpova committed Nov 9, 2017
    Configuration menu
    Copy the full SHA
    13406d0 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2017

  1. Fix the log levels, and skip outputting debug.

    Olessia Karpova committed Nov 10, 2017
    Configuration menu
    Copy the full SHA
    4cc81ea View commit details
    Browse the repository at this point in the history