Skip to content
Permalink
Tree: 4656becafb
Commits on Mar 18, 2019
  1. sys/openbsd: add chflags descriptions

    mptre authored and dvyukov committed Mar 18, 2019
  2. sys/linux: Add rfkill description for Linux

    a13xp0p0v authored and dvyukov committed Mar 17, 2019
  3. Update setup_ubuntu-host_qemu-vm_x86-64-kernel.md

    skrtbhtngr authored and dvyukov committed Mar 6, 2019
    change chown to chmod
Commits on Mar 17, 2019
  1. pkg/vcs: fix too long line

    dvyukov committed Mar 17, 2019
    Also restructure because we point to this file from syzbot docs.
  2. dashboard/app, syz-ci: bisection support

    dvyukov committed Mar 2, 2019
    This adds bulk of support for bisection to dashboard/app and syz-ci:
    - APIs to send bisection jobs and accept results
    - syz-ci logic to execute bisection jobs
    - formatting of emails with results
    - showing of results on dashboard
    
    Some difficulties we have to overcome:
    - since linux is frequently build/boot broken, lots of bisections are inconclusive,
      need to present such results too
    - git bisect is poorly suitable for automation, have to resort to output parsing (is output stable?)
    - git bisect turns out to fail (exit with non-0 status) when bisection is inconclusive
      (multiple potential cause commits)
    - older syzkaller revisions can't be built with newer (broken) kernel header, e.g.:
      ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’
    - newer compilers produce more warnings and break old syzkaller builds, e.g.:
      kvm.S.h:6:12: error: ‘kvm_asm64_vm86’ defined but not used [-Werror=unused-const-variable=]
    - figuring relevant emails to CC from a commit is non-trivial:
      besides commit author, there can be some emails in commit tags, or not,
      which tags to use is an interesting question (some may include irrelevant emails)
      we can also run get_maintainers.pl on the commit, but this can produce too wide
      list if commit touches lots of files, it can also produce too small list,
      and then we need to resort to blame
    - for inconclusive bisection we probably don't need to include emails referenced
      in the commits (there can be too many of these commits)
    - need to be careful to exclude own syzbot email from commit CC list,
      now syzbot emails are referenced in some commits (Reported-by/Tested-by/etc)
      (can cause some kind of infinite recursion)
    - lots of commits reference stable mailing list,
      we should not include it in CC because it's referenced for backports rather then bug reports
    - since we add new Bug entity fields which we use in queries,
      whole datastore need to be upgrades to add the new field to index
    - we must not discard the crash that was used for bisection
      (treat it as a reported crash)
    - bisection results need 2 forms of reports:
      one when we add bisection results to already reported bug
      another when we report a bug first time with bisection results
    - when reporting a bug with bisection results we need to use the crash
      that was used for bisection
    - some fraction of bisections will probably fail with various errors
      and we will need some mechanism to retry bisection after the root cause is resolved
      this is not implemented yet
    - linux-next is problematic for 2 reasons:
      fix bisection can't possibly run on linux-next as commits are not reachable from HEAD
      lots of commits are missing in linux-next (even in linux-next-history)
      e.g. we have some c63e9e91a254a52 which is now missing in linux-next/linux-next-history
    - older kernels can't be build with fresh gcc/binutils/perl/make/glibc
      for now we have to stop at v3.9 (this only requires switching gcc several times along the way)
    - kernels past v4.11 do not build with gcc 7 and 8 (undefined reference to `____ilog2_NaN')
    - v4.1 and back have only compiler-gcc5.h
    - v3.17 and back have only compiler-gcc4.h
    - v3.6 and back do not have make olddefconfig
    - compat socket calls can't be bisected past "x86/entry/syscalls: Wire up 32-bit
      direct socket calls" (v4.10) because of
      https://syzkaller.appspot.com/bug?id=b5b150e322d5f48c869bcf1528cdbee08d1421cb
    - v2.6.28 and below does not work with modern make:
      *** mixed implicit and normal rules: deprecated syntax
    - v3.8 build fails:
      Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at kernel/timeconst.pl line 373.
      kernel/Makefile:134: recipe for target 'kernel/timeconst.h' failed
    - make 3.81 works for v2.6.28.
      3.81 almost works with current HEAD, you need to run make twice because first run spuriously fails with:
    - v2.6.28 with gcc-4.9.4 broken with:
      include/linux/kvm.h:240:9: error: duplicate member ‘padding’
    - but even defconfig fails:
      VDSO    arch/x86/vdso/vdso.so.dbg
      gcc: error: elf_x86_64: No such file or directory
      gcc: error: unrecognized command line option ‘-m’
      It seems that we also need old binutils.
    - for v3.8 and below we need perl-5.14.4.
      Unfortunately this or any manually built perl doesn't work for later kernels:
      Can't locate strict.pm in @inc
    - kernels starting from 4.14 and older are boot broken:
      https://lkml.org/lkml/2018/9/7/648
    - kernels older than 4.12 are broken during netdev setup
      (fixed by commit 675c8da049fd6556eb2d6cdd745fe812752f07a8)
    
    Update #501
  3. vm/qemu: detect boot errors faster

    dvyukov committed Mar 14, 2019
    Currently we try to ssh into the machine for 10 minutes
    even if it crashed right away. Make qemu exit on kernel panic
    and stop ssh'ing when qemu exits.
    Handling bad kernels fast is actually important for bisection.
    
    Update #501
  4. pkg/instance: fix boot error detection

    dvyukov committed Mar 16, 2019
    Currently we truncate output up to rep.EndPos after unexpected reboot.
    But report sets EndPos to the _last_ report in output,
    so if there are any other errors they are all skipped after
    truncation to EndPos. Truncate just one line instead.
  5. syz-fuzzer: communicate image testing errors to caller

    dvyukov committed Mar 16, 2019
    Prefix image testing errors with BUG: so that they are detected
    by whoever is analyzing the output. Otherwise currently they are
    dignosed as just "lost connection to test machine".
  6. dashboard/app: add handler for config migration

    dvyukov committed Mar 15, 2019
    updateBugReporting adds missing reporting stages to bugs in a single namespace.
    Use with care. There is no undo.
    This can be used to migrate datastore to a new config with more reporting stages.
    This functionality is intentionally not connected to any handler.
    Before invoking it is recommented to stop all connected instances just in case.
  7. syz-ci: fix formatting of error message

    dvyukov committed Mar 13, 2019
    PrependContext prepends the string rather than format.
  8. tools/syz-testbuild: add utility for kernel build testing

    dvyukov committed Mar 13, 2019
    syz-testbuild tests kernel build/boot on releases as it will be done by pkg/bisect.
    This allows to ensure that, for example, a change to kernel config won't break
    build/boot on older releases and consequently won't break bisection process.
    The binary needs to run under root because it creates images.
    The kernel checkout given to the tool will be cleaned and used for in-tree builds.
    Example invocation:
    
    sudo syz-testbuild -kernel_src $LINUX_CHECKOUT \
    	-config dashboard/config/upstream-kasan.config \
    	-sysctl dashboard/config/upstream.sysctl \
    	-cmdline dashboard/config/upstream-apparmor.cmdline \
    	-userspace $WHEEZY_USERSPACE \
    	-bisect_bin $BISECT_BIN
    
    A suitable wheezy userspace can be downloaded from:
    https://storage.googleapis.com/syzkaller/wheezy.tar.gz
    A set of binaries required for bisection (older compilers) can be downloaded from:
    https://storage.googleapis.com/syzkaller/bisect_bin.tar.gz
  9. pkg/vcs: reset repo more

    dvyukov committed Mar 13, 2019
    If we cherry-pick some fixes during bisection
    we need to "git reset" repo before "git bisect reset".
    Otherwise it will fail. Reset repo in more points.
    
    Update #501
  10. pkg/vcs: try to work around massive linux kernel build/boot breakages

    dvyukov committed Mar 12, 2019
    Linux kernel is frequently build/boot broken. Even on release tags.
    Everything past 4.15 is broken with our config for multiple reasons.
    This makes bisection process almost infeasible.
    Try to work around some breakages by disabling configs are we go back in time.
    
    Update #501
  11. vm/qemu: s/Boot/boot/

    dvyukov committed Mar 12, 2019
    Make Boot lower-case to not confuse readers that it's somehow exported.
    It is not. Pure implementation detail.
  12. pkg/bisect: various improvements

    dvyukov committed Mar 12, 2019
    A bunch of improvements after more wide bisection testing.
    Improve logging.
    Support returning several commits for inconclusive bisection.
    Return Report with the final crash.
    Remove code that was moved to pkg/vcs.
    
    Update #501
  13. syz-ci: add flag that allows to not start managers

    dvyukov committed Mar 12, 2019
    For bisection testing.
    
    Update #501
  14. syz-ci: ignore job errors on shutdown

    dvyukov committed Mar 12, 2019
    Ctrl+C can kill a child process which will cause an error.
    Ignore such errors, it's better to retry after restart then
    to report a false failure.
    
    Update #501
  15. syz-ci: move GOPATH/GOROOT setup to main

    dvyukov committed Mar 12, 2019
    GOPATH/GOROOT are also needed for jobs.
    Updater can be disabled with -autoupdate=0,
    so move GOPATH/GOROOT setup to main.
    
    Update #501
  16. dashboard/app: make dashapi.BugReport more self-contained

    dvyukov committed Mar 12, 2019
    Extend BugReport with few fields that are needed by all reportings anyway.
    This allows to not create and fill an almost identical object to pass to template.
    
    Update #501
  17. dashboard/app: load crash in loadJob test helper

    dvyukov committed Mar 12, 2019
    Will be useful for bisection testing.
    
    Update #501
  18. dashboard/app: use pkg/html to format text emails

    dvyukov committed Mar 12, 2019
    Use pkg/html helpers to format text emails.
    Shorten short hashes to 8 char while we are here,
    the length used by git log --oneline.
    Tidy up tests a bit.
    
    Update #501
  19. dashboard/app: add markCrashReported helper

    dvyukov committed Mar 12, 2019
    Factor out markCrashReported out of incomingCommandTx.
    Use it when creating patch testing jobs.
    Will be needed for bisection jobs as well.
    
    Update #501
  20. dashboard/app: add Log text entity

    dvyukov committed Mar 12, 2019
    To store bisection logs.
    
    Update #501
  21. dashboard/app: rename CommitInfo to Commit

    dvyukov committed Mar 12, 2019
    Because why?
  22. pkg/vcs: refactor bisection support

    dvyukov committed Mar 12, 2019
    In preparation for syz-ci bisection:
    - move bisection function into a separate interface
      they look out of place in vcs.Repo because most OSes
      don't implement it and most users don't case
    - extract author name and more CC emails for commits
    - move linux-specific PreviousReleaseTags into linux.go
    - fix inconclusive bisection (more than 1 potential commits)
    - add tests fr bisection
    - add maintainers returned from get_maintainers.pl for commits
      that don't have enough emails (e.g. only author email)
    
    Update #501
  23. dashboard/app: modernize tests

    dvyukov committed Mar 12, 2019
    Use expectOK for checking errors more.
    Use pollEmailBug helper more.
    Add expectNE and use it.
  24. pkg/log: add VerboseWriter

    dvyukov committed Mar 12, 2019
    Helper io.Writer that logs on the specified verbosity level.
    Will be useful for bisection logging in syz-ci.
    
    Update #501
  25. pkg/instance: use -fpermissive -w when building old syzkaller

    dvyukov committed Mar 12, 2019
    Since we can be building very old revisions for bisection here,
    make the build as permissive as possible.
    Newer compilers tend to produce more warnings also kernel headers may be broken, e.g.:
    ebtables.h:197:19: error: invalid conversion from ‘void*’ to ‘ebt_entry_target*’
    
    Update #501
  26. pkg/osutil: return output with errors from Run

    dvyukov committed Mar 12, 2019
    Turns out git bisect fails to communicate some outcomes (multiple potential commits),
    and the only way to understand what happened is parsing plain English output.
    Extracting the output from the error is quite unhandy.
    So return it with the error.
    
    Update #501
  27. pkg/osutil: fix disabling of sandbox

    dvyukov committed Mar 12, 2019
    If sandboxing is disabled don't unshare net namespace too.
    
    Update #501
  28. pkg/html: add more helper function

    dvyukov committed Mar 12, 2019
    Add optlink, formatKernelTime, formatCommitTableTitle.
    Will be useful to format bisection emails.
    
    Update #501
  29. vendor/github.com/google/go-cmp: update to HEAD

    dvyukov committed Mar 12, 2019
    To pick up fix for:
    google/go-cmp#103
  30. docs: describe syzbot bisection

    dvyukov committed Mar 17, 2019
    Update #501
Older
You can’t perform that action at this time.