feat(packing): labels to refer to other parts of the result

[bennofs]

This is useful if you're building payloads that needs to refer to locations
relative to itself such as rop chains or printf format strings.

Example usage:

In [1]: x = Label()
In [2]: y = Label()
In [3]: fit(lambda: {
   ...:     0x0: [
   ...:         "abc" + str(y.begin()),
   ...:         y.mark("data"),
   ...:         x.begin()
   ...:     ],
   ...:     0x18: [ "abc", x.mark("x" + str(x.size())) ],
   ...: })
Out[3]: 'abc4data\x1b\x00\x00\x00daaaeaaafaaaabcx2'

There is no documentation yet because I first want to make sure that the general idea is ok.

Other (future) possible enhancement:

• provide a special label for the current "part", so you can do:

  flat(lambda: [
    p64(Label.here() + 8), # refers to "some data"
    "some data",
  ])

• provide label adapters that can be used for printf for example:

  x = PrintfArg(Label(), 8) # 8 is the printf stack offset
  
  flat(lambda: [
    "{}$s".format(x.arg(0)), # refers to "$9" -> 0xdeadbeef
    x.mark([
      0xdeadbeef,
      ....
    ])
  ])

  Or they could add a base address for rop chain data etc.

[bennofs]

The current "iterate until fixpoint" approach does not terminate in general, but I think the cases where it won't terminate won't occur in practice. If I am not mistaken, it should only happen if you use exponentials or other "non-basic" arithmetic functions.

An example which doesn't terminate (this actually takes very long to evaluate, we may want to limit MAX_ITERATIONS a bit more)

In [3]: flat(lambda: [
   ...:     str(10**l.begin()),
   ...:     l.mark("data")
   ...: ])

Also, if there are multiple solutions, it is not guarranteed to find the shortest one (depends on initial label values):

In [16]: flat(lambda: [
    ...:     str(10**(l.begin()-1)),
    ...:     l.mark("data")
    ...: ])
Out[16]: '100data'

(1data is a shorter solution)

But again, I think these cases are not relevant in practice. Most of the time, I expect only linear calculations to be performed on label values for which the algorithm should work nicely.

[zachriggle]

Sorry for leaving this for a while.

I think that the idea is useful -- but have a couple suggestions.

First, this ends up being a good bit of functionality on top of fit. I think a new function should be defined, rather than adding the functionality to fit directly.

Second, the interface is a little bit clunky -- x.size() instead of len(x), x.begin() rather than just x, and x.mark(...) rather than just x(...). Alternately

Third, this functionality definitely needs docs and especially some unit tests.

I've tried to clean up the implementation a bit in #1063

[zachriggle]

Here's some of your examples which I've adapted to the different style used in my Pull Request.

x = fit.Label()
y = fit.Label()
fit({
    0: lambda: ['abc' + str(y.address),
                y('data'),
                x.address],
    0x18: lambda: ['abc', x("x" + str(len(x)))]
})
# 'abc4data\x1b\x00\x00\x00daaaeaaafaaaabcx2'

The feature Label.here() that you referred to can be emulated pretty easily by placing an empty label (no contents) followed by the its address in a lambda.

here = fit.Label()
flat([here, lambda: here.address + 8, "some data"], address=0x10000)
# '\x08\x00\x01\x00some data'

I'm not sure what your goal is with the PrintfArg bit, but you can mock it up pretty easily:

class PrintfArg(fit.Label):
    def __init__(self, index):
        super(PrintfArg, self).__init__()
        self.index = index

    def arg(self, offset):
        return self.index + offset

x = PrintfArg(8)
flat([
    lambda: "%{}$s".format(x.arg(0)),
    lambda: x([0xdeadbeef])
])
# '%8$s\xef\xbe\xad\xde'

The exponential cases are still an issue, but I don't think it's tractable to solve.

l = fit.Label()
flat([
     lambda: str(10**l.address),
     lambda: l("data")
])
# doesn't finish

The "sub-optimal solution" isn't something I'm concerned about, and the example to show it off is a bit contrived. Both solutions arrive at the same answer. The reason for this solution appearing is a side-effect of the initial value of l.begin().

l = fit.Label()
flat([
     lambda: str(10**(l.address-1)),
     lambda: l("data")
])
# '100data'

Overall, the changes are:

1. Cosmetic
   • .address instead of .begin()
   • () instead of .mark()
   • len() instead of .size()
2. Exception-driven rather than object-tracking
   • When the data or address on any object changes, an exception is raised so that everything else can be re-calculated
3. Permissive of lambdas pretty much everywhere
   • Rather than a single lambda that returns everything, you can use smaller / individual lambdas

>>> a = fit.Label()
>>> offset = lambda: (len(a) / 4)
>>> contents = lambda: a("Dynamic contents.  Offset %i, Length %i" % (a.address, len(a)))
>>> fit({offset: contents})
'aaaabaaacDynamic contents.  Offset 9, Length 38'

[TethysSvensson]

I have an alternative implementation of this somewhere. Not sure if it's actually better. I'll try digging it up.

[TethysSvensson]

After looking at my old code, I am not sure that it has a higher level of usability than the one implemented in #1063. The only feature my code had that was not here was the ability to easily put something on an alignment boundary.

[bennofs]

Closing this in favor of #1063