Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch for TF 2.3.1 #43358

Merged
merged 20 commits into from Sep 20, 2020
Merged

Patch for TF 2.3.1 #43358

merged 20 commits into from Sep 20, 2020

Commits on Sep 18, 2020

  1. [tflite] Don't check for buffers on every subgraph.

    Buffers in the model are allocated globally, hence it makes sense to check for
    their presence only once (O(1)) instead of on every subgraph (O(n)).
    
    PiperOrigin-RevId: 323677724
    Change-Id: I2da0c381093006828cc4c80f03dec8a917782861
    mihaimaruseac committed Sep 18, 2020
    Copy the full SHA
    2369d14 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2020

  1. [tflite] Validate segment ids for segment_sum.

    Segment identifiers in segment_sum should be in a 1-D tensor of same size as the first dimension of the input. The values of the tensor should be integers from {0, 1, 2, ... k-1}, where k is the first dimension of the input. The segment identifiers must not contain jumps and must be increasing.
    
    See https://www.tensorflow.org/api_docs/python/tf/math#Segmentation as the source for these constraints.
    
    PiperOrigin-RevId: 332510942
    Change-Id: I898beaba00642c918bcd4b4d4ce893ebb190d869
    mihaimaruseac committed Sep 19, 2020
    Copy the full SHA
    00c7ed7 View commit details
    Browse the repository at this point in the history
  2. [tflite] Test for kTfLiteOptionalTensor in GetInput.

    `GetInput`, `GetVariableInput` and `GetOutput` all fail to check for the case where `node->inputs->data[index]` is the special `kTfLiteOptionalTensor` value (-1) which then causes `context->tensors[node->inputs->data[index]]` to read from invalid memory location.
    
    This fix makes `GetInput` and related return `nullptr` in those cases, asking the caller to check for `nullptr`. This is better than having `GetOptionalInputTensor` and `GetOptionalOutputTensor` (does not exist but could be added) as using the patched `GetInput` in error would be caught by a sanitizer test in the default optimized build (due to the `-fsanitize=null` option).
    
    PiperOrigin-RevId: 332512190
    Change-Id: Iabca54da2f2de02b6ece3c38b54f76d4277d689e
    mihaimaruseac committed Sep 19, 2020
    Copy the full SHA
    42ed6ac View commit details
    Browse the repository at this point in the history
  3. [tflite] Make GetOptionalInputTensor the same as GetInput.

    With the previous change, there is no more need for two separate APIs. We would deprecate `GetOptionalInputTensor` in the future.
    
    PiperOrigin-RevId: 332513386
    Change-Id: Id7110271c25ebd6126ad8c82a493e37e0e0756b3
    mihaimaruseac committed Sep 19, 2020
    Copy the full SHA
    7e283f9 View commit details
    Browse the repository at this point in the history
  4. [tflite] Ensure inputs and outputs don't overlap.

    If a model uses the same tensor for both an input and an output then this can result in data loss and memory corruption. This should not happen.
    
    PiperOrigin-RevId: 332522916
    Change-Id: If0905b142415a9dfceaf2d181872f2a8fb88f48a
    mihaimaruseac committed Sep 19, 2020
    Copy the full SHA
    094329d View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2020

  1. [tflite] Ensure input tensors don't have nullptr buffers.

    A crafted TFLite model can force a node to have as input a tensor backed by a `nullptr` buffer. That is, by carefully changing the buffer index in the flatbuffer serialization, we can force the TFLite interpreter to consider a read-only tensor to be a read-write one and assume that there is an operator that has this tensor as output, writing to it and allocating memory before the tensor is used as input. If this does not happen, we get memory corruption.
    
    PiperOrigin-RevId: 332524692
    Change-Id: I57ef175152a29020af9ab041dc959e5631dce40f
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    1a506ae View commit details
    Browse the repository at this point in the history
  2. [tflite] Ensure MatchingDim does not allow buffer overflow.

    We check in `MatchingDim` that both arguments have the same dimensionality, however that is a `DCHECK` only enabled if building in debug mode. Hence, it could be possible to cause buffer overflows by passing in a tensor with larger dimensions as the second argument. To fix, we now make `MatchingDim` return the minimum of the two sizes.
    
    A much better fix would be to return a status object but that requires refactoring a large part of the codebase for minor benefits.
    
    PiperOrigin-RevId: 332526127
    Change-Id: If627d0d2c80a685217b6e0d1e64b0872dbf1c5e4
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    cd671a9 View commit details
    Browse the repository at this point in the history
  3. [tflite] Ensure ResolveAxis properly handles negative inputs.

    In Python, a list `l` of length `n` allows indexing with negative indices, `l[i]`. The only constraint is that `n + i` becomes positive. Code in `ResolveAxis` assumes the constraints and only checks it using a `DCHECK`. But the macro is a no-op in non-debug builds and that can result in reading from negative offsets (buffer underflows).
    
    PiperOrigin-RevId: 332530683
    Change-Id: I464e073fee618054ae3719a3679739007bb3f3bc
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    b392591 View commit details
    Browse the repository at this point in the history
  4. Validate NodeDefs from FunctionDefLibrary of a GraphDef.

    We already validated `NodeDef`s from a `GraphDef` but missed validating those from the `FunctionDefLibrary`. Thus, some maliciously crafted models could evade detection and cause denial of service due to a `CHECK`-fail.
    
    PiperOrigin-RevId: 332536309
    Change-Id: I052efe919ff1fe2f90815e286a1aa4c54c7b94ff
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    9a64529 View commit details
    Browse the repository at this point in the history
  5. Fix bad import

    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    b98674e View commit details
    Browse the repository at this point in the history
  6. Copy the full SHA
    0fde760 View commit details
    Browse the repository at this point in the history
  7. Validate data_splits for tf.StringNGrams.

    Without validation, we can cause a heap buffer overflow which results in data leakage and/or segfaults.
    
    PiperOrigin-RevId: 332543478
    Change-Id: Iee5bda24497a195d09d122355502480830b1b317
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    892d5e5 View commit details
    Browse the repository at this point in the history
  8. Prevent segfault in GetSessionHandle{,V2}.

    In eager mode, session state is null.
    
    PiperOrigin-RevId: 332548597
    Change-Id: If094812c2e094044220b9ba28f7d7601be042f38
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    ce90127 View commit details
    Browse the repository at this point in the history
  9. Prevent format string vulnerability in tf.strings.as_string.

    The `printf` format specifier only allows `#`, `0`, `-`, `+` and space as flag characters. Others are interpreted as width/precision/length modifier or conversion specifiers. If a character does not fit into any of these sets `printf` just displays it.
    
    Also add a test suite for `tf.strings.as_string`. Also fix the issue where the flag character was used only if width was specified.
    
    PiperOrigin-RevId: 332553548
    Change-Id: Ie57cf2a7c14d1a36097642794c14329db669bbba
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    0315fa7 View commit details
    Browse the repository at this point in the history
  10. Prevent int64 to int truncation in Shard API usage.

    The function argument in `Shard` must be a function of two `int64` arguments. However, we are passing in a function with two `int` arguments. Thus, for large workloads, these arguments get truncated from positive `int64` values to negative `int` ones, resulting in a buffer out of bounds write.
    
    PiperOrigin-RevId: 332557334
    Change-Id: I236c9a2e7f53580e520571da8ba941a3aa9fa0b5
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    cb1422c View commit details
    Browse the repository at this point in the history
  11. Prevent integer truncation from 64 to 32 bits.

    The `tensorflow::Shard` functions last argument must be a 2 argument function where both arguments are `int64` (`long long`, 64 bits). However, there are usages where code passes in a function where arguments are `int` or `int32` (32 bits). In these cases, it is possible that the integer truncation would later cause a segfault or other unexpected behavior.
    
    PiperOrigin-RevId: 332560414
    Change-Id: Ief649406babc8d4f60b3e7a9d573cbcc5ce5b767
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    42783a6 View commit details
    Browse the repository at this point in the history
  12. Fix multiple vulnerabilities in tf.raw_ops.*CountSparseOutput.

    Also add tests for these API points, both for the happy paths and for the vulnerable ones.
    
    PiperOrigin-RevId: 332563222
    Change-Id: Ib3b52116a83a134c2e742a7c66e5e956db8fba05
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    b307563 View commit details
    Browse the repository at this point in the history
  13. Fix heap buffer overflow in tf.raw_ops.SparseFillEmptyRowsGrad.

    Also add tests as they were lacking
    
    PiperOrigin-RevId: 332566071
    Change-Id: I44277578e26ff5fb3fdb0dcbba6e91b2ec3e7859
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    156872d View commit details
    Browse the repository at this point in the history
  14. Fix multiple vulnerabilities in tf.experimental.dlpack.to_dlpack.

    We have a use after free caused by memory coruption, a segmentation fault caused by memory corruption, several memory leaks and an undefined behavior when taking the reference of a nullptr.
    
    PiperOrigin-RevId: 332568894
    Change-Id: Ife0fc05e103b35325094ae5d822ee5fdea764572
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    d8c69c2 View commit details
    Browse the repository at this point in the history
  15. Fix undefined behavior in tf.raw_ops.Switch in eager mode.

    PiperOrigin-RevId: 332578058
    Change-Id: I9727571d2f21476b10d8aa27c1b7176564b76ac9
    mihaimaruseac committed Sep 20, 2020
    Copy the full SHA
    92d5b97 View commit details
    Browse the repository at this point in the history