Skip to content
Tree: 60562a1d09
Commits on Dec 8, 2018
  1. sys/freebsd: use a better description of msg_flags fields

    tuexen authored and dvyukov committed Dec 8, 2018
    As suggested by Dmitry us a better description of the msg_flags
    field, which is only used to provide information from the
    kernel to the application for recvmsg() calls. This means that
    the value provided is basically ignored.
  2. tools/syz-trace2syz/proggen: fix name shadowing

    dvyukov committed Dec 8, 2018
  3. tools/syz-trace2syz/proggen: comment blank import

    dvyukov committed Dec 8, 2018
    golint suggests to comment all blank imports.
    But actually we don't need whole sys, we can import only sys/linux.
    Also rename target var to prevent name shadowing.
  4. prog: rename ProgGen to Builder

    dvyukov committed Dec 8, 2018
    golint suggests that "prog.Prog" is a bad naming
    because everything in prog package is ProgSomething.
    Rename to Builder, "prog.Builder" sounds right.
Commits on Dec 7, 2018
  1. tools/syz-trace2syz/proggen: remove currentStraceArg

    dvyukov committed Dec 7, 2018
    It's used only by 2 functions: genSockaddrNetlink and genIfrIfru.
    Majority of functions just accept the straceType as argument,
    which looks like a much more appropriate way to pass an argument to a function.
    Amusingly, both functions already accept and use the straceType as argument.
  2. tools/syz-trace2syz/proggen: unexport and refactor Context

    dvyukov committed Dec 7, 2018
    1. Unexport Context, it's not meant for callers.
    2. Unexport all Context fields.
    3. Make all function Context methods.
  3. tools/syz-trace2syz/proggen: convert tests to table format

    dvyukov committed Dec 7, 2018
    This has number of advantages:
    1. Tests are readable and writable.
       The current checks [1] are neither.
    2. Tests are much more compact.
    3. Tests verify all aspects rather than just
       1 aspect of the resulting program.
    4. Tests are much less fragile.
    5. Any diffs in the results will be more clearly visible.
    switch a := p.Calls[1].Args[0].(type) {
    case *prog.ResultArg:
    	if a.Res != p.Calls[0].Ret {
    switch a := p.Calls[1].Args[0].(type) {
    case *prog.ResultArg:
    	pipeSecondFd := p.Calls[0].Args[0].(*prog.PointerArg).Res.(*prog.GroupArg).Inner[1]
    	if a.Res != pipeSecondFd {
    write := p.Calls[len(p.Calls)-2]
    inotifyRmWatch := p.Calls[len(p.Calls)-1]
    switch a := write.Args[0].Type().(type) {
    case *prog.ResourceType:
    	if a.TypeName != "fd" {
    		t.Fatalf("expected first argument of write to have type fd, got: %s", a.TypeName)
    	t.Fatalf("first argument of write is not resource type: %s", a.Name())
    switch a := inotifyRmWatch.Args[1].(type) {
    case *prog.ResultArg:
    	b := a.Type().(*prog.ResourceType)
    	if b.TypeName != "inotifydesc" {
    		t.Fatalf("expected second argument of inotify_rm_watch to have type inoitfydesc, got: %s", b.TypeName)
    	if a.Res != p.Calls[2].Ret {
    		t.Fatalf("inotify_rm_watch's second argument should match the result of inotify_add_watch.")
    sockaddr, ok := a.(*prog.PointerArg).Res.(*prog.GroupArg)
    if !ok {
    	t.Fatalf("%s", a.Type().Name())
    ipv4Addr, ok := sockaddr.Inner[2].(*prog.UnionArg)
    if !ok {
    	t.Fatalf("expected 3rd argument to be unionArg, got %s", sockaddr.Inner[2].Type().Name())
    optName := ipv4Addr.Option.Type().FieldName()
    if !strings.Contains(optName, "rand") {
    	t.Fatalf("expected ip option to be random opt, got: %s", optName)
    ip, ok := ipv4Addr.Option.(*prog.ConstArg)
    if !ok {
    	t.Fatalf("ipv4Addr option is not IntType")
    if ip.Val != expectedIp {
    	t.Fatalf("parsed != expected, %d != %d", ip.Val, expectedIp)
  4. tools/syz-trace2syz/proggen: fix vma allocation

    dvyukov committed Dec 7, 2018
    There are 2 bugs:
    1. We always allocate 1 page, even if use more.
    2. VMA addresses are not aligned, so most mmap-like functions fail with EINVAL.
    The added test currently panics with "unaligned vma address".
  5. tools/syz-trace2syz: add go-fuzz fuzzer

    dvyukov committed Dec 7, 2018
    Inputs like "2__R" or "3_F	T.3.3l" make
    traze2syz hang infinitely consuming all machine memory.
    Need to fix all crashes over time.
  6. tools/syz-trace2syz: start adding proper error handling

    dvyukov committed Dec 7, 2018
    log.Fatal is not the proper way to handle errors.
    It does not allow to write good tests, fuzzers
    and utilities that crash all the time.
  7. tools/syz-trace2syz/proggen: add ParseFile function

    dvyukov committed Dec 7, 2018
    Current code structuring has 2 problems:
    1. parsing anything with proggen requires complex multistep dance including
     - parsing data with parser
     - walking the resulting tree manually and calling proggen on each
     - then for each context
       - calling FillOutMemory (unclear why it's not part of parsing)
       - calling prog.Finalize
       - checking is the program is not too large
    All of this duplicated across trace2syz and tests.
    And any new tests or fuzzers we will write will need to duplicate
    all of this logic too.
    2. As the result of this structuring, lots of proggen guts
    and implementation details are exposed.
    While none of the callers are actually intersted in Context details,
    they are not interested in Context itself whatsoever.
    What every caller wants is "here is data to parse, give me programs".
    Add such function.
  8. tools/syz-trace2syz/parser: remove Filename from TraceTree

    dvyukov committed Dec 7, 2018
    We already printed file name of the trace in parseTraces,
    no need to print it again and again.
    Consequently we don't need Filename in TraceTree.
    If needed, caller can always log it before parsing,
    or pass along with the TraceTree.
  9. tools/syz-trace2syz: use short variable declaration syntax

    dvyukov committed Dec 7, 2018
    Use short variable declaration syntax where possible.
    Move declarations closer to usages.
  10. tools/syz-trace2syz/parser: use []byte instead of string for file con…

    dvyukov committed Dec 7, 2018
    If we are handling whole files, it's more efficient to use []byte.
    string is not really meant to hold large amounts of data.
  11. tools/syz-trace2syz: adding missing copyright headers

    shankarapailoor authored and dvyukov committed Dec 6, 2018
    Adding missing copyright headers to return_cache.go and context.go
  12. pkg/build/openbsd: require kernel config files as configuration data

    blackgnezdo authored and dvyukov committed Dec 6, 2018
    Previously the config was generated directly, but testing multiple
    configurations makes this cumbersome going forward. This makes
    kernel_config a mandatory parameter.
  13. sys/targets: some syscalls on OpenBSD does need defines

    mptre authored and dvyukov committed Dec 6, 2018
    Some syscalls on OpenBSD violates the ordinary SYS_ prefix convention. This is
    an exhaustive enumeration of the deviations.
    Regression introduced in commit 88746fd ("pkg/csource: use defines from
    sys/syscall.h on *bsd").
Commits on Dec 6, 2018
  1. tools/syz-trace2syz/proggen/return_cache.go: format string mismatch

    blackgnezdo authored and dvyukov committed Dec 6, 2018
  2. prog: add Prog.Finalize

    dvyukov committed Dec 6, 2018
    Prog.Finalize combines assignSizesCall, SanitizeCall and validate.
    Intended for users who build own programs,
    so that we don't need to expose all individual methods.
  3. prog: export Type.DefaultArg

    dvyukov committed Dec 6, 2018
    It's effectively exported anyway.
    So export it the proper way.
  4. tools/syz-trace2syz: skip 2 more syscalls

    dvyukov committed Dec 6, 2018
    These set_robust_list and set_tid_address are issued by glibc
    for every process/thread start.
    Normal programs don't use them and it's unlikely we build
    something interesting with them (e.g. we won't get real robust list in memory).
    Skip them.
  5. tools/syz-trace2syz: tidy up code

    dvyukov committed Dec 6, 2018
    Lots of assorted changes mainly converting code to idiomatic Go
    and replacing code with equivalent, but shorter code.
  6. pkg/db: provide helper function for database creation

    dvyukov committed Dec 6, 2018
    This is needed for both tools/syz-db and tools/syz-trace2syz.
    Also, remove code to resolve SHA1 collisions.
    Also, don't set db version as we actually want to minimize
    and smash these programs like anything else
    (not minimizing nor smashing them is only useful during tool testing).
  7. tools/syz-trace2syz: merge config package into proggen

    dvyukov committed Dec 6, 2018
    Since we now have only single variable there,
    it does not seem to deserve a separate package.
  8. tools/syz-trace2syz: add tool to convert strace output to programs

    shankarapailoor authored and dvyukov committed Dec 6, 2018
    * fixing weird merge error
    * fixing presubmit
    * fixing presubmit
    * removing parsing code because of -Xraw option
    * fix presubmit
    * update
    * deleting vma_call_handlers as we are currently skipping most vma calls. This simplifies memory_tracker as we don't need to keep track of vma allocations
    * removing custom handling of bpf_instruction union
    * removing ifconf parsing
    * update
    * removed all expression types and replaced them with constant types. removing ipv6_addr parsing while -Xraw is getting fixed. Removing constants.go
    * removing ipv6 parsing
    * presubmit
    * moving direction check from ipv4_addr out to genUnion
    * removing code that parses kcov
    * removing redundant test
    * removing custom code in generate unions to fill ipv4_addr
    * proggen: changing order of imports to make external packages import first
    fixing presubmit
    * changing log messages to lower case to be consistent with other packages.
    * removing pointer type and simplifying memory_tracker
    removing comment
    * moving context and return_cache to seaparate files
    * deleting default argument generation when we should probably throw an error
  9. Update

    dvyukov committed Dec 6, 2018
    add newer clang compiler
  10. Update

    dvyukov committed Dec 6, 2018
Commits on Dec 5, 2018
  1. pkg/csource: exclude linux/arm64 tests

    dvyukov committed Dec 5, 2018
    I think I misinterpreted the error that episodically happens on ci:
    collect2: error: ld terminated with signal 11 [Segmentation fault], core dumped
    compiler invocation: aarch64-linux-gnu-gcc [-Wall -Werror -O1 -g -o /tmp/syz-executor570589071 -pthread -DGOOS_linux=1 -DGOARCH_arm64=1 -x c - -static]
    as OOM, but they all involve aarch64-linux-gnu-gcc:
    So I guess the problem can be with the arm64 toolchain that just crashes randomly.
  2. sys/freebsd: remove linux specific include files for ICMP

    tuexen authored and dvyukov committed Dec 5, 2018
  3. sys/freebsd: improve TCP tests

    tuexen authored and dvyukov committed Dec 5, 2018
    * sys/freebsd: improve TCP tests
    Add missing TCP socket options for FreeBSD.
    * sys/freebsd: improve TCP tests
    Add socket option description for TCP_FASTOPEN.
You can’t perform that action at this time.