Skip to content
Tree: 65ed24721e
Commits on Dec 7, 2018
  1. tools/syz-trace2syz/proggen: remove currentStraceArg

    dvyukov committed Dec 7, 2018
    It's used only by 2 functions: genSockaddrNetlink and genIfrIfru.
    Majority of functions just accept the straceType as argument,
    which looks like a much more appropriate way to pass an argument to a function.
    Amusingly, both functions already accept and use the straceType as argument.
  2. tools/syz-trace2syz/proggen: unexport and refactor Context

    dvyukov committed Dec 7, 2018
    1. Unexport Context, it's not meant for callers.
    2. Unexport all Context fields.
    3. Make all function Context methods.
  3. tools/syz-trace2syz/proggen: convert tests to table format

    dvyukov committed Dec 7, 2018
    This has number of advantages:
    1. Tests are readable and writable.
       The current checks [1] are neither.
    2. Tests are much more compact.
    3. Tests verify all aspects rather than just
       1 aspect of the resulting program.
    4. Tests are much less fragile.
    5. Any diffs in the results will be more clearly visible.
    switch a := p.Calls[1].Args[0].(type) {
    case *prog.ResultArg:
    	if a.Res != p.Calls[0].Ret {
    switch a := p.Calls[1].Args[0].(type) {
    case *prog.ResultArg:
    	pipeSecondFd := p.Calls[0].Args[0].(*prog.PointerArg).Res.(*prog.GroupArg).Inner[1]
    	if a.Res != pipeSecondFd {
    write := p.Calls[len(p.Calls)-2]
    inotifyRmWatch := p.Calls[len(p.Calls)-1]
    switch a := write.Args[0].Type().(type) {
    case *prog.ResourceType:
    	if a.TypeName != "fd" {
    		t.Fatalf("expected first argument of write to have type fd, got: %s", a.TypeName)
    	t.Fatalf("first argument of write is not resource type: %s", a.Name())
    switch a := inotifyRmWatch.Args[1].(type) {
    case *prog.ResultArg:
    	b := a.Type().(*prog.ResourceType)
    	if b.TypeName != "inotifydesc" {
    		t.Fatalf("expected second argument of inotify_rm_watch to have type inoitfydesc, got: %s", b.TypeName)
    	if a.Res != p.Calls[2].Ret {
    		t.Fatalf("inotify_rm_watch's second argument should match the result of inotify_add_watch.")
    sockaddr, ok := a.(*prog.PointerArg).Res.(*prog.GroupArg)
    if !ok {
    	t.Fatalf("%s", a.Type().Name())
    ipv4Addr, ok := sockaddr.Inner[2].(*prog.UnionArg)
    if !ok {
    	t.Fatalf("expected 3rd argument to be unionArg, got %s", sockaddr.Inner[2].Type().Name())
    optName := ipv4Addr.Option.Type().FieldName()
    if !strings.Contains(optName, "rand") {
    	t.Fatalf("expected ip option to be random opt, got: %s", optName)
    ip, ok := ipv4Addr.Option.(*prog.ConstArg)
    if !ok {
    	t.Fatalf("ipv4Addr option is not IntType")
    if ip.Val != expectedIp {
    	t.Fatalf("parsed != expected, %d != %d", ip.Val, expectedIp)
  4. tools/syz-trace2syz/proggen: fix vma allocation

    dvyukov committed Dec 7, 2018
    There are 2 bugs:
    1. We always allocate 1 page, even if use more.
    2. VMA addresses are not aligned, so most mmap-like functions fail with EINVAL.
    The added test currently panics with "unaligned vma address".
  5. tools/syz-trace2syz: add go-fuzz fuzzer

    dvyukov committed Dec 7, 2018
    Inputs like "2__R" or "3_F	T.3.3l" make
    traze2syz hang infinitely consuming all machine memory.
    Need to fix all crashes over time.
  6. tools/syz-trace2syz: start adding proper error handling

    dvyukov committed Dec 7, 2018
    log.Fatal is not the proper way to handle errors.
    It does not allow to write good tests, fuzzers
    and utilities that crash all the time.
  7. tools/syz-trace2syz/proggen: add ParseFile function

    dvyukov committed Dec 7, 2018
    Current code structuring has 2 problems:
    1. parsing anything with proggen requires complex multistep dance including
     - parsing data with parser
     - walking the resulting tree manually and calling proggen on each
     - then for each context
       - calling FillOutMemory (unclear why it's not part of parsing)
       - calling prog.Finalize
       - checking is the program is not too large
    All of this duplicated across trace2syz and tests.
    And any new tests or fuzzers we will write will need to duplicate
    all of this logic too.
    2. As the result of this structuring, lots of proggen guts
    and implementation details are exposed.
    While none of the callers are actually intersted in Context details,
    they are not interested in Context itself whatsoever.
    What every caller wants is "here is data to parse, give me programs".
    Add such function.
  8. tools/syz-trace2syz/parser: remove Filename from TraceTree

    dvyukov committed Dec 7, 2018
    We already printed file name of the trace in parseTraces,
    no need to print it again and again.
    Consequently we don't need Filename in TraceTree.
    If needed, caller can always log it before parsing,
    or pass along with the TraceTree.
  9. tools/syz-trace2syz: use short variable declaration syntax

    dvyukov committed Dec 7, 2018
    Use short variable declaration syntax where possible.
    Move declarations closer to usages.
  10. tools/syz-trace2syz/parser: use []byte instead of string for file con…

    dvyukov committed Dec 7, 2018
    If we are handling whole files, it's more efficient to use []byte.
    string is not really meant to hold large amounts of data.
  11. tools/syz-trace2syz: adding missing copyright headers

    shankarapailoor authored and dvyukov committed Dec 6, 2018
    Adding missing copyright headers to return_cache.go and context.go
  12. pkg/build/openbsd: require kernel config files as configuration data

    blackgnezdo authored and dvyukov committed Dec 6, 2018
    Previously the config was generated directly, but testing multiple
    configurations makes this cumbersome going forward. This makes
    kernel_config a mandatory parameter.
  13. sys/targets: some syscalls on OpenBSD does need defines

    mptre authored and dvyukov committed Dec 6, 2018
    Some syscalls on OpenBSD violates the ordinary SYS_ prefix convention. This is
    an exhaustive enumeration of the deviations.
    Regression introduced in commit 88746fd ("pkg/csource: use defines from
    sys/syscall.h on *bsd").
Commits on Dec 6, 2018
  1. tools/syz-trace2syz/proggen/return_cache.go: format string mismatch

    blackgnezdo authored and dvyukov committed Dec 6, 2018
  2. prog: add Prog.Finalize

    dvyukov committed Dec 6, 2018
    Prog.Finalize combines assignSizesCall, SanitizeCall and validate.
    Intended for users who build own programs,
    so that we don't need to expose all individual methods.
  3. prog: export Type.DefaultArg

    dvyukov committed Dec 6, 2018
    It's effectively exported anyway.
    So export it the proper way.
  4. tools/syz-trace2syz: skip 2 more syscalls

    dvyukov committed Dec 6, 2018
    These set_robust_list and set_tid_address are issued by glibc
    for every process/thread start.
    Normal programs don't use them and it's unlikely we build
    something interesting with them (e.g. we won't get real robust list in memory).
    Skip them.
  5. tools/syz-trace2syz: tidy up code

    dvyukov committed Dec 6, 2018
    Lots of assorted changes mainly converting code to idiomatic Go
    and replacing code with equivalent, but shorter code.
  6. pkg/db: provide helper function for database creation

    dvyukov committed Dec 6, 2018
    This is needed for both tools/syz-db and tools/syz-trace2syz.
    Also, remove code to resolve SHA1 collisions.
    Also, don't set db version as we actually want to minimize
    and smash these programs like anything else
    (not minimizing nor smashing them is only useful during tool testing).
  7. tools/syz-trace2syz: merge config package into proggen

    dvyukov committed Dec 6, 2018
    Since we now have only single variable there,
    it does not seem to deserve a separate package.
  8. tools/syz-trace2syz: add tool to convert strace output to programs

    shankarapailoor authored and dvyukov committed Dec 6, 2018
    * fixing weird merge error
    * fixing presubmit
    * fixing presubmit
    * removing parsing code because of -Xraw option
    * fix presubmit
    * update
    * deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations
    * removing custom handling of bpf_instruction union
    * removing ifconf parsing
    * update
    * removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go
    * removing ipv6 parsing
    * presubmit
    * moving direction check from ipv4_addr out to genUnion
    * removing code that parses kcov
    * removing redundant test
    * removing custom code in generate unions to fill ipv4_addr
    * proggen: changing order of imports to make external packages import first
    fixing presubmit
    * changing log messages to lower case to be consistent with other packages.
    * removing pointer type and simplifying memory_tracker
    removing comment
    * moving context and return_cache to seaparate files
    * deleting default argument generation when we should probably throw an error
  9. Update

    dvyukov committed Dec 6, 2018
    add newer clang compiler
  10. Update

    dvyukov committed Dec 6, 2018
Commits on Dec 5, 2018
  1. pkg/csource: exclude linux/arm64 tests

    dvyukov committed Dec 5, 2018
    I think I misinterpreted the error that episodically happens on ci:
    collect2: error: ld terminated with signal 11 [Segmentation fault], core dumped
    compiler invocation: aarch64-linux-gnu-gcc [-Wall -Werror -O1 -g -o /tmp/syz-executor570589071 -pthread -DGOOS_linux=1 -DGOARCH_arm64=1 -x c - -static]
    as OOM, but they all involve aarch64-linux-gnu-gcc:
    So I guess the problem can be with the arm64 toolchain that just crashes randomly.
  2. sys/freebsd: remove linux specific include files for ICMP

    tuexen authored and dvyukov committed Dec 5, 2018
  3. sys/freebsd: improve TCP tests

    tuexen authored and dvyukov committed Dec 5, 2018
    * sys/freebsd: improve TCP tests
    Add missing TCP socket options for FreeBSD.
    * sys/freebsd: improve TCP tests
    Add socket option description for TCP_FASTOPEN.
Commits on Dec 4, 2018
  1. sys/freebsd: improve udp tests

    tuexen authored and dvyukov committed Dec 4, 2018
    Add support for the UDP_ENCAP. Add required includes and
    remove the Linux specific ones.
  2. sys: remove socketpair for AF_INET and AF_INET6

    tuexen authored and dvyukov committed Dec 4, 2018
    * OpebBSD: remove socketpair() for AF_INET and AF_INET6.
    socketpair() is only supported on AF_UNIX.
    * NetBSD: remove socketpair() for AF_INET and AF_INET6.
    socketpair() is only supported for AF_UNIX.
    * FreeBSD: remove socketpair() for AF_INET and AF_INET6.
    socketpair() only supports AF_UNIX.
    * Linux: remove socketpair for AF_INET and AF_INET6.
    socketpair only supports AF_UNIX.
    * Autogenerated files.
    These are manually generated for all platforms you are not
    running on. FreeBSD in this case.
    * executor: rebase.
    * sys/freebsd: rebase.
    * sys/linux: use AF_UNIX based socketpair for nbd.
    This was suggested by Dmitry.
    Fixes #845
  3. tools/ shut off pagination in ddb

    blackgnezdo authored and dvyukov committed Dec 4, 2018
Commits on Dec 3, 2018
  1. sys/freebsd: add UDP-Lite descriptions

    tuexen authored and dvyukov committed Dec 3, 2018
  2. pkg/csource: reduce short tests

    dvyukov committed Dec 3, 2018
    pkg/csource test gets OOM-killed on travis:
    Add several measures:
     - set GOMAXPROCS=1 to restrict parallel processes
     - remove -g from compiler invocation
     - reduce set of tests run in short mode to compensate for GOMAXPROCS=1
     - also reduce set of tests in full mode as they timeout now
You can’t perform that action at this time.