Permalink
Commits on Nov 12, 2018
  1. changes an error message in case of empty file (#896)

    gitoleg authored and ivg committed Nov 12, 2018
    * changes an error message in case of empty file
    
    This PR changes an error message if bap is using with an emptyt file:
    ```
    $ touch /tmp/foo.txt
    $ bap /tmp/foo.txt
    Failed to create a project: ("File format is not supported: expected executable, library or kernel module"
      "Raised at file \"src/import0.ml\" (inlined), line 234, characters 22-32\
     \nCalled from file \"src/error.ml\" (inlined), line 9, characters 14-30\
     \nCalled from file \"src/or_error.ml\", line 70, characters 17-32\
     \nCalled from file \"lib/bap/bap_project.ml\", line 264, characters 47-54\
     \nCalled from file \"src/or_error.ml\", line 62, characters 9-15\
     \n")
    
    ```
    
    * edited message
    
    * better output + check input file
    
    * added strip
    
    * refactored a little
    
    * updated testsuite
Commits on Nov 8, 2018
  1. adds integration with ida 7 (#893)

    gitoleg committed Nov 8, 2018
    * ida 7 integartion
    
    * refactored a little
    
    * Ida 7 launches only in terminal version
    
    * work on error messages
    
    * refactoring
    
    * typo fix
Commits on Oct 30, 2018
  1. escapes strings before generating OGRE document (#891)

    gitoleg authored and ivg committed Oct 30, 2018
    * escape everything but digit and letters
    
    * fix comment
Commits on Oct 10, 2018
  1. ensures static linking with llvm >= 4.0 (#889)

    gitoleg committed Oct 10, 2018
    it turned out that after `llvm-4.0` a default linking mode is `shared`,
    and it's not what we want. e.g. debian package will be broken, as `bap`
    will be depend from `LLVM.so`.
    
    our previous request for libraries now returns an extremly short list:
    ```
    ubuntu@ubuntu-xenial:~$ llvm-config-4.0 --libs
    -lLLVM-4.0
    ```
    which is just shared library.
    
    So what we can do, is to use a new flag for llvm-config:
    ```
    ubuntu@ubuntu-xenial:~$ llvm-config-4.0 --link-static --libs
    -lLLVMLTO -lLLVMPasses -lLLVMObjCARCOpts -lLLVMMIRParser ...
    
    ```
    Here it is!
Commits on Oct 9, 2018
  1. adds `llvm 7.0` support (#887)

    gitoleg committed Oct 9, 2018
    * adds `llvm 7.0` support
    
    * edited CHANGES file
Commits on Oct 6, 2018
  1. silenced warnings from ours deprecations (#885)

    gitoleg committed Oct 6, 2018
    * silenced warnings from ours deprecations
    
    This PR silences warnings from bap code during compilation of
    bap itself. Also, it turned out that few modules in `bap.mli`
    were deprecated in a way that doesn't emit warnings at all:
    
    Just for the record, the next doesn't deprecate `module A`,
    because according to the documentation for attributes of level 3:
    > They are not attached to any specific node in the syntax tree
    
    ```
    module A = struct
      [@@@deprecated "[since ...]"]
      ...
    end
    ```
    
    And the next works as expected, although in this case we have
    do the deprecation in the end of module, which makes it less
    readable. But anyway, looks like we don't have any other choice.
    
    ```
    module A = struct
      ...
    end [@@deprecated "[since ...]"]
    ```
    
    * fixed the documentation
Commits on Oct 5, 2018
  1. updates rtl in mips and powerpc plugins (#884)

    gitoleg committed Oct 5, 2018
    the removed code was written mainly for support our `run` tests
    and no longer needed after adding bil plugin, as now everything works
    out of the box.
Commits on Oct 4, 2018
  1. handles correctly format change in the cache module (#881)

    gitoleg authored and ivg committed Oct 4, 2018
    * prevent bap from fail if cache entry does'n exists
    
    * made it more sophisticated
    
    * updated
    
    * refactored a little
  2. removed an unexhaustive pattern matching warning (#882)

    gitoleg authored and ivg committed Oct 4, 2018
    ```
    File "plugins/x86/x86_tools_mem.ml", line 58, characters 6-322:
    Warning 8: this pattern-matching is not exhaustive.
    Here is an example of a case that is not matched:
    ((`YMM0|`YMM3|`XMM10|`XMM1|`YMM4|`XMM15|`XMM6|`YMM1|`YMM11|`XMM12|`YMM15|
     `XMM13|`YMM10|`YMM9|`YMM13|`XMM3|`XMM9|`XMM11|`YMM5|`XMM2|`YMM7|`YMM14|
     `YMM8|`YMM2|`XMM0|`XMM5|`XMM7|`XMM4|`XMM8|`XMM14|`YMM12|`YMM6),
    _)
    
    ```
    
    So here we make an expression for effective address, which doesn't
    involve SSE registers.
    
    Also, let's quote
    `Intel® 64 and IA-32 architectures software developer’s manual, volume 1,
    Section 3.7.5 (Specifying an Offset)`:
    
    ```
    The offset part of a memory address can be specified directly as a static value (called a displacement) or through an address computation made up of one or more of the following components:
    
    Displacement — An 8-, 16-, or 32-bit value.
    Base — The value in a general-purpose register.
    Index — The value in a general-purpose register.
    Scale factor — A value of 2, 4, or 8 that is multiplied by the index value.
    ```
Commits on Sep 19, 2018
  1. Update CHANGES.md

    gitoleg committed Sep 19, 2018
  2. updates version and changes (#876)

    gitoleg committed Sep 19, 2018
    * updates changes file
  3. fixes builds on travis (#875)

    gitoleg committed Sep 19, 2018
    * updates travis env
Commits on Sep 14, 2018
  1. few fixes in llvm backend (#873)

    gitoleg authored and ivg committed Sep 14, 2018
    Looks like that we didn't build bap with `llvm-3.4` for a while, so
    there were couple of minor bugs there. This PR fixes them.
Commits on Sep 7, 2018
  1. makes the main `bap` brancher a default one (#870)

    gitoleg authored and ivg committed Sep 7, 2018
    Since we still don't merge brancher sources, we
    need to set a default one, which will be used in case
    if many branchers available.
  2. improves relocations handling (#768)

    gitoleg authored and ivg committed Sep 7, 2018
    * added a relocatabale reconstructor
    
    * work in progress
    
    * it's working!!
    
    * refactored relocator
    
    * refactoring
    
    * added Bap_relocator
    
    Technicaly, we may add this module to plugin,
    but will also have to update api pligin dependencies
    in register_pass function, since api should be applied
    after all relocations resolving
    
    * work in progress
    
    * it's working!
    
    * made the relocatable bracnher the default one!
    
    * lot's of debug code in Rel_reconstructor.
    
    Last commit for him
    
    * removed rel_reconstructor, rel_fact
    
    * refactoring bap_relocator
    
    * refactoring
    
    * removed wrong dependency
    
    * adds a cfg to IR lifting function
    
    We still have a problem, that bil doesn't reflect
    jump instruction destinations in case of relocations
    presence. This leads to wrong IR program.
    
    One of possible solutions is to add a whole program cfg
    into sema lift functions for destinations search.
    
    Once we did it, we don't need to think about local relocations
    at all - only about external symbols
    
    * added cfg again
    
    * renamed Bap_relocator to Bap_inject_externals
    
    * added to symtab `find_call` and `add_call`
    
    * refactoring
    
    * identation
    
    * fixed bug with calls discovering
    
    * resolved jump as calls with no return
    
    * just fix doc
    
    * moved everything toa plugin
    
    * removed unused + refactored
    
    * refactoring
    
    * removed unused
    
    * lot's of changes
    
    Added to Symtab functions `add_callee` and `find_callee`.
    It became possible to add a name of callee side in Reconstructor.
    It's useful for detection of external calls while lifting to IR.
    
    * wip
    
    * made it workable
    
    added to sema_lift a lookup of callees of
    indirect calls
    
    * updated testsuite
    
    * review accepted
    
    * just remove unnecessary changes
    
    * edited typo
    
    * fixed docs again
    
    * refactored brancher
    
    * updated testsuite
    
    * fixed a bug with cross section call
    
    let's harden a resolving of external functions and
    check indirect calls against addresses in symtab too:
    if address is in symtab, then such call can't be external
    
    * refactoring
    
    * updated documentation
    
    * fixed predicate for resolving jumps
  3. adds const propagation (#830)

    gitoleg authored and ivg committed Sep 7, 2018
    * adds const propagation
    
    * refactored a little
    
    * reworked, added Simpl tests
    
    * renaming
    
    * restored lost associativity
    
    * adds few optimizations for distributive operations
    
    * removed division cases, added plus/minus cases
    
    * refactoring
    
    * more tests, refactoring
    
    * added more tests on binops
    
    * removed debug code
    
    * simpifications on cast and extract + tests
    
    * verbose tests a bit
    
    * refactoring
    
    * applyed new simpl cases recursively
    
    * reworked tests a little
    
    * added more tests and cases
    
    * added a few more tests
    
    * bugs fixed
    
    * minor bug fixed
    
    * updated on helpers
    
    * updated tests
    
    * wrote a simplification for like terms
    
    * added a separate module for my consts stuff
    
    * remmoved unused code
    
    * just few stylistic changes
    
    * refactoring
    
    * added a few forgotten cases
    
    * refactored
    
    * returned back a couple of rec call of exp
    
    * fixed minor bug
    
    * added a guard in pretify
    
    * added a draft of bil analysis plugin
    
    * refactoring + thoughts about pipeline
    
    * rectified a bass registration/invocation
    
    * refactored bil-pipe
    
    * just rewrote select_basses
    
    * minor details
    
    * corrected mc
    
    * just stylistic changes
    
    * fiexed a bug in pretifying exps
    
    * added one more case
    
    * removed self#mapping
    
    * fixed minor bug
    
    * added a guard, just in case
    
    * rectify applyng extract and cast
    
    * added some concat/extract/cast cases
    
    * updated cast
    
    * refactoring
    
    * just added type for readability
    
    * worklist cons propagation
    
    * refactoring
    
    * fixed bug in cast simpl
    
    * refactoring
    
    * more refactoring
    
    * just few improves
    
    * bug fixed + test added
    
    * moved const prop to a plugin
    
    * just few doc string + stylistic changes
    
    * moved everything to a plugin
    
    * fixed copy propagation
    
    * refactoring
    
    * more refactoring
    
    * just edited graph building a little
    
    * refactoring, bug fixing
    
    * ended with graph approach
    
    * redone consts
    
    * renamed
    
    * small portion of refactoring
    
    * rewritten bil dead code elimination
    
    * refactored
    
    * refactored simplifications
    
    * fixed tests a little
    
    * refactoring .. will need more tests
    
    * bug fixed
    
    * canceled cmdline changed
    
    * refactored
    
    * just few refactoring
    
    * more refactoring
    
    * important fix in dead code elimination,refactoring
    
    * bug fixed
    
    * rectified symtab a little
    
    * updating dead code elimiination
    
    * work on reconstructor
    
    * rectified reconstructor
    
    so there is only one iteration over the whole graph
    
    * updating dead code elimination
    
    * few variants of dead code elimination
    
    * type infer fix
    
    * merge with master stage 2
    
    * symc testsuite
    
    * removed memoization
    
    * deleted unused files
    
    * updating names
    
    * added memo
    
    * updated bil dead_code_elimination
    
    * refactored a little
    
    * added `passes` option, remade registration
    
    So now, library provides interface for registration
    of various bil analysis. And a plugin selects these
    analysis
    
    * just refactored a little
    
    * removed fold_consts from brancher
    
    * a bit updated docs
    
    * refactoring
    
    * refactoring + bug fixing + fixpoint for all
    
    1) fixed a minor bug in normalization process, and prevent
    normalizeds store expressions from further normalization.
    
    2) fixed `fixpoint` in order to increase probability of loop
    detection
    
    3) implemented a simple live/dead analysis on bil plugin,
    that affects only to virtual variables
    
    4) refactored cmdline arguments and docstring for bil plugin
    
    * fixed live anaysis for while loops
    
    * few major changes, see description below
    
    1) moved everything to the bap library
    2) keep_ites flag for normalization doesn't exists anymore
    
    * added few tests, will add more
    
    * updated tests
    
    * added a propagation of unknowns + refactoring
    
    * fixed tests
Commits on Sep 4, 2018
  1. enables bap-elf (#866)

    gitoleg authored and ivg committed Sep 4, 2018
    * enables bap-elf
    
    in order to use `bitstring.ppx`, we should not use `ppx_driver` and `ppx_custom` predicates according to
    bistring META file. And ppx_jane can't be used too. That's why we just
    omit `bap_elf` diretory in our tags file.
    
    * added lost dependency
    
    * updated flags
Commits on Aug 31, 2018
Commits on Aug 30, 2018
  1. implements deadcode-elimination pass optimization (#857)

    gitoleg committed Aug 30, 2018
    * improves dead-code-elimination
    
    This PR intends to speed up the dead-code-elimination plugin.
    At first, it adds cache, so every next launch will use a result
    of a first one.
    
    At second, and this need to be discussed IRL, this reduces a number
    of SSA form calculation, that subsequently speed up the whole plugin
    
    * bug fixed
    
    * bug fixed again
    
    * updated after review
    
    Sorry, indented the buffer, so a bit more effort could be required
    for review.
    
    changes:
    1) consts propagation and dead code elimination separated one from
       another. Previously, the number of times we called `propagate_consts`
       was the same as number of times we did dead code elimination.
       Theoreticly speaking, these changes could kill more unused
       code, since we propagate consts until reaching of a fixpoint,
       and only then eliminate dead code.
    2) caching takes in account results of consts propagation too
    3) refactoring
    
    * refactoring and bug fixing
    
    This PR brings back interleaving of constant propagation
    and dead code elimination. Also optimizes constant propagation
    in part of computing set of virtual variables for propagation:
    it's done only once now.
    
    * const propagation in ssa form
    
    We run const propagation in ssa form, so we don't need to track once defined
    variable, since they are obviously defined only once in ssa.
    Added a cmdline flag for propagating consts in real variables too,
    i.e. in registers.
    
    * adds a level of optimization
    
    Now, set of variables that are affectd by analysis could
    be defined through the comamnd line
    
    * refactored and reduced memory usage
    
    This PR reduces a memory usage by applying analysis to
    each subroutine separatly, so there are not multiply copies
    of the same subroutine in memory, like it was before.
    
    * added attribute for updated terms
    
    * refactoring
    
    * fixed a bug in sub's digest
    
    Sub name was't taken in account, so digest could be the same
    for different (but similar) subroutines
    
    * renamed dead-code-elimination to optimization
    
    * updated testsuite
    
    * bug fixed
    
    * feqw updates after review
    
    * edited doc
    
    * issue a warning on --no-dead-code-elimination
  2. saves cache index in bin_prot (#865)

    gitoleg committed Aug 30, 2018
    * saves cache's index with bin_io
    
    Previously we used `sexp` for storing index, but it turned out
    that it is very slowly. And in this PR we save index with `bin_io`,
    that solves the problem.
    
    * fix ida-servide lock
    
    we forgot to close lock after unlocking
    
    * few updates after review
    
    * remove tmp in case of troubles
Commits on Aug 29, 2018
  1. fixes calls search in reconstructor (#863)

    gitoleg authored and ivg committed Aug 29, 2018
    * updates roots in reconstructor
    
    Actually, this PR just restores previous behaviour,
    that was lost due to lack of tests and comments.
    
    So this PR:
    1) extends a list of roots with addresses of such CFG nodes,
       that don't have any input edges. The effect is noticeable in
       case of pure code input, since the list of roots is empty
       in such case, and therefore the reconstructor will create
       just an empty symtab.
    2) adds comments in code to avoid of loosing such behaviour in
       future.
    3) updated documentation for reconstructor in `bap.mli`, since
       it looks like quite outdated
    
    * updated testsuite
    
    * updated comments
  2. deprecates X86_env.nums in correct place (#864)

    gitoleg authored and ivg committed Aug 29, 2018
    just moved `[@@deprecated]` after `X86_env.nums`.
    Previously we had in compile time the next warning
    ```
    Warning 3: deprecated: CPU.r
    [since 2018-01] user `r` instead
    ```
    i.e. a new definition was deprecated, not a previous one.
Commits on Aug 15, 2018
  1. fixes the bug that was producing unreachable blocks (#856)

    gitoleg authored and ivg committed Aug 15, 2018
    * Removes wrong roots between stages of diasasm
    
    Some significant problems can happen if one of our rooters give
    us some wrong roots. We can even miss instructions and build a
    wrong cfg. So this PR removes wrong roots on the early stages
    of the disassembler, so they don't influence of the late ones.
    
    * updated
Commits on Aug 14, 2018
  1. optimizes reconstructor, symtab, and brancher performance (#855)

    gitoleg authored and ivg committed Aug 14, 2018
    * rectifies reconstructor, symtab and brancher
    
    This PR rectifies reconstructor, symtab and brancher with a respect to
    performance, without adding new behaviour or breaking of existed one.
    
    Rewrote it and reduced a number of iterations over nodes/edges of cfg
    
    There was a bit inefficient implementation of `add_symbol` function,
    so every addition of a symbol led to a filter of the whole table,
    although there are enough info to reduce such calls.
    
    Just a small fix that check if an instruction has jumps at all before
    subsequent call of `fold_consts` that could be heavy for some of
    instructions
    
    * updated after review
    
    * refactored
Commits on Aug 10, 2018
  1. Fix deprecated args (#854)

    gitoleg authored and ivg committed Aug 10, 2018
    * warn user in case of using deprecated flags
    
    also warns if deprecated parameter is used as flag
    
    * renaming
    
    * renamed
  2. cleans /tmp/ida every time ida is called (#853)

    gitoleg authored and ivg committed Aug 10, 2018
    * cleans /tmp/ida every time ida is called
    
    fix #720
    
    This PR removes dump files from `/tmp/ida` before any ida launch
    
    * refactored
    
    * added info message
    
    * renamed
    
    * updated after review
Commits on Aug 3, 2018
  1. updates travis env (#851)

    gitoleg authored and ivg committed Aug 3, 2018
    * this PR updates travis env
    
    After last changes in `ocaml-ci-scripts`, a default version
    of `opam` is `2.0`, that breaks our builds.
    The simpliest fix is just to set an environment variable
    `OPAM_VERSION=1.2.2`.
    
    Also we can add theirs scripts directly to `bap`, like it
    was some time ago, but I still suggest to do it only when
    there wan't be any easy way to fix some issue from thier
    side.
    
    * added one more env var
    
    * cleaned up
    
    * returned an old version of ci-scripts
Commits on May 31, 2018
  1. added powerpc tests (#838)

    gitoleg authored and ivg committed May 31, 2018
Commits on May 23, 2018
  1. updated testsuite (#837)

    gitoleg committed May 23, 2018
    pcmpistr is not skipped anymore in veri tests
  2. Fix packed x86 (#836)

    gitoleg authored and ivg committed May 23, 2018
    * fixed pcmpeq, pcmpgt, pmin, pmax ..
    
    * added tests to oasis
    
    * fixed PCMP*STR* instructions
    
    so bap can produce bil for them again
    
    * refactored pcmpstr
    
    * edited tests for pcmp
    
    * added pmaxsb, pmaxsd insns
    
    * updated tests
Commits on May 9, 2018
  1. fixes bug in reconstructor (#834)

    gitoleg authored and ivg committed May 9, 2018
    This PR fixes a BUG in reconstructor, where we actually ignored
    lots of calls
Commits on Apr 18, 2018
Commits on Apr 10, 2018
  1. adds mangling of duplicated subroutines (#813)

    gitoleg authored and ivg committed Apr 10, 2018
    * first approach to mangling duplicated subroutines
    
    Implemented like an iteration over existed subroutines
    
    * refactoring
    
    * work in progress: all checks are on program type
    
    * made the program type private
    
    * changes after review
    
    * reimplemented all with Map
    
    * placed everything in Program module
    
    * refactored
    
    * added a check if nothing should be changed
Commits on Apr 9, 2018
  1. rectifies powerpc lifter (#809)

    gitoleg authored and ivg committed Apr 9, 2018
    * updated model
    
    * started to fix branch insns
    
    * replaced most of wrong << using
    
    * fixed doubleword using in rotate
    
    * typo
    
    * refactored a couple of instructions