Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sanitizer coverage feedback evolution support part2 #47

Merged
merged 36 commits into from
Jan 14, 2016

Commits on Jan 3, 2016

  1. ANDROID: SIGABRT not important signal

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 3, 2016
    Configuration menu
    Copy the full SHA
    97ac3bf View commit details
    Browse the repository at this point in the history
  2. SANCOV: Upgrade data collection

    Clang sanitizer coverage (sancov) data parsing functions. Supported methods:
     * raw unified data (preferred method)
     * individual data per executable/DSO (not preferred since lots of data lost if  instrumented code exits
       abnormally or with sanitizer unhandled signal (common in Android OS)
    
    For raw-unpack method a global (shared across workers) Trie is created for the chosen
    initial seed and maintained until seed is replaced. Trie nodes store the loaded (as exposed
    from *.sancov.map file) execs/DSOs from target application using the map name as key. Trie node
    data struct (trieData_t) maintains information for each instrumented map including a bitmap with
    all hit relative PC addresses (realPC - baseAddr to circumvent ASLR). Map's bitmap is updated while
    new areas on target application are discovered based on absolute elitism implemented at
    fuzz_sanCovFeedback().
    
    For individual data files a PID (fuzzer's thread) based filename search is performed to identify
    all files belonging to examined execution. This method doesn't implement yet bitmap runtime data
    to detect newly discovered areas. It's mainly used so far as a comparison metric for raw-unpack method
    and stability check for sancov experimental features such as coverage counters:
    http://clang.llvm.org/docs/SanitizerCoverage.html
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 3, 2016
    Configuration menu
    Copy the full SHA
    a16f70f View commit details
    Browse the repository at this point in the history
  3. SANCOV: Update dynFile counter after blacklist check

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 3, 2016
    Configuration menu
    Copy the full SHA
    5a31141 View commit details
    Browse the repository at this point in the history
  4. ANDROID: Disable ASan abort_on_error

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 3, 2016
    Configuration menu
    Copy the full SHA
    18891ab View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2016

  1. Merge branch 'sancov' into sancov-dev

    * sancov:
    anestisb committed Jan 6, 2016
    Configuration menu
    Copy the full SHA
    e97e944 View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2016

  1. ANDROID: ASan abort code change

    SIGABRT is not a monitored signal (thus 'abort_on_error' is missing crashes when set)
    for Android OS since it produces lots of useless crashes due to way Android process
    termination hacks work. Safest option is to register & monitor one of user signals.
    SIGUSR2 is used for sanitizer fuzzing in Android, although might need to be changed
    if target uses it for other purposes.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    c34451a View commit details
    Browse the repository at this point in the history
  2. SANCOV: Thread safety improvements

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    1fd10c7 View commit details
    Browse the repository at this point in the history
  3. SANCOV: Comments update

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    b78cf60 View commit details
    Browse the repository at this point in the history
  4. Merge branch 'sancov' into sancov-dev

    * sancov:
      make depend
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    45cc1a2 View commit details
    Browse the repository at this point in the history
  5. ANDROID: ASan exitcode fixes

    ASan exitcode flag used in Android due to unmonitored
    SIGABRT, doesn't raise any signals. Thus needs to be
    treated at the target pid exit code level.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    39bf6cf View commit details
    Browse the repository at this point in the history
  6. SANCOV: Fix newPC counter bug when initial runs

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    a6458d9 View commit details
    Browse the repository at this point in the history
  7. SANCOV: Don't mangle 1st iteration

    In order to have accurate coverage data to work against
    the first iteration of a new seed pickup is not mangled. This
    will save the coverage bitmaps of original input.
    
    In case of multiple worker threads, only one picks this tasks
    and keeps a lock until finished, blocking other threads from
    continuing fuzzing.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    ac05480 View commit details
    Browse the repository at this point in the history
  8. Comments update

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    995b47b View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2016

  1. LINUX: Cleanup sanitizer flags

    Also updated crash data analysis when based on
    exit codes instead of raised signal.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 8, 2016
    Configuration menu
    Copy the full SHA
    3e0ea96 View commit details
    Browse the repository at this point in the history
  2. SANCOV: Fix memory leaks & off-by-1 OOB

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 8, 2016
    Configuration menu
    Copy the full SHA
    267f0d8 View commit details
    Browse the repository at this point in the history
  3. LINUX: Refactor sanitizer flags

    Add global string buffers to store the dynamically constructed
    sanitizer flags based on invocation arguments. Buffers are
    initialized once during LINUX arch init, avoiding performance
    overhead on each child spawn.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 8, 2016
    Configuration menu
    Copy the full SHA
    61b5ab1 View commit details
    Browse the repository at this point in the history
  4. LINUX: Make stack hash #MajorFrames dynamic

    For sanitizer enabled targets with 'abort_on_error' set, the
    number of major frames needs to increased since the top
    7-9 frames are occupied with sanitizer internal symbols. Is
    sanitizer enabled targets major frames are increased to 14
    preventing possible unique crashes from getting lost.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 8, 2016
    Configuration menu
    Copy the full SHA
    56ccf1d View commit details
    Browse the repository at this point in the history
  5. LINUX: Add missing definition

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 8, 2016
    Configuration menu
    Copy the full SHA
    a1b0a65 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2016

  1. INDENT: Small edits so that OS X indent can be used

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    ccdf28c View commit details
    Browse the repository at this point in the history
  2. SANCOV: Log bitmap overflow for big target bins

    Log error of bitmap overflow so that error can be tracked
    and increase size if necessary for specific targets.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    58c45d2 View commit details
    Browse the repository at this point in the history
  3. FILES: Fix memory leaks

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    3a8e16f View commit details
    Browse the repository at this point in the history
  4. LINUX: Add ASan report parsing

    Crashing PC, address, type of error & stack frames
    parsed from ASan report files. Generated reports and
    crash filenames have been updated keeping format
    compatibility with signal detected crashes.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    e520810 View commit details
    Browse the repository at this point in the history
  5. LINUX: Don't save reports if abort enabled

    For Linux arch where abort_on_error is enabled, don't
    save report files since they're not parsed thus never
    deleted (polluting the workdir). Also fixed a small typo
    in MSAN flags.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    3a6de2c View commit details
    Browse the repository at this point in the history
  6. LINUX: Fix ASan report parse memory leak

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    ea0bcb6 View commit details
    Browse the repository at this point in the history
  7. LINUX: Add blacklist & ignoreAddr filterers to ASan exitcode crashes

    Since both crash address & call stack hashes are available, apply
    filters for ignore addresses & blacklisted hashes.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    c28e5ed View commit details
    Browse the repository at this point in the history
  8. SANCOV: counter updates

    Increase crashes counter maintained for each new
    seed pick-up from initial input corpus when ASan
    report parsing method is triggered to process
    detected crashes.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    6b43204 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2016

  1. LINUX: Remove dead macro

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    7a53b07 View commit details
    Browse the repository at this point in the history
  2. SANCOV: Rename variables

    Renamed data structs, counters & printed information
    in order to be technically more accurate based on
    instrumentation techniques offered by clang sanitizers.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    56e360f View commit details
    Browse the repository at this point in the history
  3. SANCOV: Remove perf timer used for dev

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    7739272 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2016

  1. LINUX: Fix ASan reports parsing bug

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 12, 2016
    Configuration menu
    Copy the full SHA
    ca556e6 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2016

  1. PERF: Empty input seed checks

    When perf feedback is enabled, user is allowed to provide
    an empty input corpus resulting into fuzzer working in
    discovery mode utilizing perf counters. Update 1st round
    of empty seed checks to be aligned with revised blocking
    thread logic.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 13, 2016
    Configuration menu
    Copy the full SHA
    6ef24e4 View commit details
    Browse the repository at this point in the history
  2. LINUX: Abstract SIGABRT monitor logic

    Instead of making SIGABRT monitor Android specific,
    define a global flag at common header to control SIGABRT
    monitor and adjust sanitizers' abort_on_error flag accordingly.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 13, 2016
    Configuration menu
    Copy the full SHA
    8e634c3 View commit details
    Browse the repository at this point in the history
  3. LINUX: Increase #MajorFrames for non-sancov runs too

    Since sanitizer flags are always set without prior knowledge
    if target is sanitizer compiled or not, always increase number
    of major frames if SIGABRT is monitored. Maybe export an
    additional argument in future, but for now seems good enough.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 13, 2016
    Configuration menu
    Copy the full SHA
    c0d2830 View commit details
    Browse the repository at this point in the history
  4. LINUX: Clean-up verifier ASan reports pollution

    Separate post crash execution actions for main workers
    and other (e.g. verifier) when exit code crash is detected.
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 13, 2016
    Configuration menu
    Copy the full SHA
    97633cc View commit details
    Browse the repository at this point in the history
  5. make indent

    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 13, 2016
    Configuration menu
    Copy the full SHA
    e680477 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2016

  1. FILES: Fix dictionary parsing bug

    Bug was introduced while resolving getdelim memory leaks
    at commit 3a8e16f
    
    Signed-off-by: Anestis Bechtsoudis <anestis@census-labs.com>
    anestisb committed Jan 14, 2016
    Configuration menu
    Copy the full SHA
    70f2fbd View commit details
    Browse the repository at this point in the history