Test for case of undefined behaviour and throw error if found #52707

elfringham · 2021-10-27T15:58:42Z

Casting to a lower precision variable type is undefined behaviour if the value being cast overflows the new type.
Relying on the outcome of such a cast lead to differing outcomes on x86 and AARCH64 in the RangeTest.testLargeStarts unit test. Detect this case before it happens and throw an exception then correct the test case to accept the new exception. In the non-eager case an InvalidArgument exception is re-thrown as a ValueError so both exceptions should be allowed in the test.

Fixes #52676

…if found

penpornk

Thank you for the PR and sorry for the delay!

PiperOrigin-RevId: 416941851 Change-Id: Iefa5a9b841b053b36f6b105cd82c9d32d5e47850

Merge pull request #52707 from elfringham:init_ops_test_fix

* Add op-determinism info to version 2.7 release notes * Move the change to the right TF release (2.7) * tflite_runtime: Loosen numpy requirement It's a common use case that using OpenCV with tflite_runtime. Since OpenCV requires minimum 1.19.3, we'd better loosen the numpy requirement of tflite_runtime. PiperOrigin-RevId: 411936173 Change-Id: Idfda9382e5af534a2fc1beffcadc6428acc12201 * [lite] Add flag TFLITE_ENABLE_HEXAGON and update build rules to only define it if compiling for arm on non-apple devices. PiperOrigin-RevId: 414330792 Change-Id: I88e534e506c03554c2cb6b079ba2bde07f1244e1 * [lite] Remove explicit #define for TFLITE_ENABLE_HEXAGON. This should be enabled by separate build target. PiperOrigin-RevId: 414750624 Change-Id: I0650e96f81e840f4f885c7f459a59d75620e3743 * Temporary disable XNNPACK on armhf Linux Python wheel build PiperOrigin-RevId: 411954096 Change-Id: I0464fe508d1a0d724799905c16acdf7a698c7f34 * tflite_runtime: Add a way to override wheel's platform name PiperOrigin-RevId: 414570169 Change-Id: I853bd6725c54595e712977f12d6c14b3d610068d * Sort signature inputs by input key instead of tensor name when loading a TF1 model in TF2. Signatures are saved using nest.flatten, which sorts the keys alphabetically for determinism. This can cause a mismatch between the lifted signature function's arg_keywords and structured_input_signature, resulting in users not being able to re-save the signature. Detailed example below. Say there is a SavedModel signature with the inputs: ``` inputs['a'] tensor_info: dtype: DT_FLOAT shape: (-1, 1) name: input_2:0 inputs['b'] tensor_info: dtype: DT_FLOAT shape: (-1, 2) name: input_1:0 ``` When this signature is loaded using the TF2 API, it becomes a concrete function with arg keywords = `['b', 'a']` (sorted by tensor name) The `structured_input_signature` becomes `([], {'b': spec_for_b, 'a': spec_for_a})` When this function is exported as a signature in a new SavedModel, the structured inputs are flattened and zipped with the arg keywords. However, the flattened `structured_input_signature` is `[spec_for_a, spec_for_b]`, since `nest.flatten` sorts the keys for determinism. This results in the wrong spec being mapped to the wrong arguments. PiperOrigin-RevId: 417889747 Change-Id: Icdce72a43033027c18f5530068f2d6fd47e8e32d * Internal change PiperOrigin-RevId: 411896058 Change-Id: Ia031058247e3cf382957a6662d3f9e1cbb481ca2 * Fix out of bound error in ReverseSequence Op shape function PiperOrigin-RevId: 411896080 Change-Id: I7e59a38e2f960886edf2b6c54ed5a84e86a9b193 * Fix out of bound access in DequantizeOp by adding check for axis < input dimension PiperOrigin-RevId: 411214268 Change-Id: I3249d2a69ddc82f182c589a3a5bbfb71543f4b29 * Fix Integer overflow error in Dequantize op shape function, by adding a bound check on axis. PiperOrigin-RevId: 412121389 Change-Id: I3088dbad9e90f9998d406b618c16694388a9dfb4 * Add negative bound check for row and column pooling_sequence in FractionalAvgPoolGrad op to avoid out of bound heap access PiperOrigin-RevId: 413837346 Change-Id: I2b86034101df31bee161abcb781755e236c7bccd * Fix integer overflow leading to divide by zero error in Unravel index kernel when dimensions product exceeds max int value. PiperOrigin-RevId: 413250052 Change-Id: I9450b6e8acecd2e881a64b882e2b7c70e8e9289a * Fix Segfault in Concat V2 shape function. PiperOrigin-RevId: 412120654 Change-Id: I3ff915faea694f9ad8b00024e9af2de9909011be * [tf.data] Set limit on number of threads used in threadpool_dataset. PiperOrigin-RevId: 410922677 Change-Id: Ib25814a99043ab10805b5d2d7088ae0e0b7b04fd * Add a check for pad width to be a positive value. PiperOrigin-RevId: 413275853 Change-Id: I261a8db9dabf5ce48a806a9e58129080c9fac619 * Fix more boosted trees vulnerabilities. Please no longer use Boosted trees inside TF. The authors of the code no longer work on this after getting their promotion and moving to other teams. * Add missing validation to sparse dense cwise ops. PiperOrigin-RevId: 415543133 Change-Id: I5baf3284e919338afb96178c468ad3d3cb0d956c * Prevent overflow in sparse dense cwise ops. PiperOrigin-RevId: 415543171 Change-Id: I22dab7c41be2121ab5efe5403ca0e2f9b7cb24b8 * Add missing validation to `AddManySparseToTensorsMap`. Sparse tensors have a set of requirements for the 3 components and not all of them were checked. PiperOrigin-RevId: 415358027 Change-Id: I96cbb672999cd1da772c22fabbd15507e32e12dc * Replace faulty overflow check with a builder for `TensorShape`. Prevents an integer overflow that was not caught before. PiperOrigin-RevId: 415381595 Change-Id: I76585ddedc912bd9f4a390aeafa8e2ced1a28863 * Add a check for Key being scalar tensor for MapStage and OrderedMapStage ops. According to documentation[1][2], key must be int64 value, but this wasn't enforced and the ops would fail with check failure for non-scalar key value. [1]https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/ordered-map-stage [2]https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/map-stage PiperOrigin-RevId: 413822112 Change-Id: I9d118faf990e6361900aa32272eff486ad9f0e2e * Fix potential divide by zero error when executing FractionalMaxPool, when pooling ratio is higher than input size for a particular dimension. PiperOrigin-RevId: 412151722 Change-Id: I06e57cbb8eca43816eff79eac264fa7aae8f7163 * Fixed model server crash: Instead of calling the constructor of TensorShape, we call the BuildTensorShapeBase factory function and check the status of the calls in sparse_slice_op. PiperOrigin-RevId: 402647675 Change-Id: Id9cba19c18ab0f16ea800c2b69d859935a54c2c7 * Add integer overflow and negative value check for some of the op kernels to avoid Check Failure in the InitDim called from constructor, before instantiating TensorShape. PiperOrigin-RevId: 414610546 Change-Id: Icc713b41ef8515d2977c4c193a7bf19dd6792ffc * Fix integer overflow error in SparseSlice op PiperOrigin-RevId: 415399219 Change-Id: Ifb3cffcb5cfacb6eab151bec6b31d7a2d7f264d7 * Add integer overflow fixes for StringNGram, SparseSliceDataset and SparseSlice ops for identified usages. PiperOrigin-RevId: 415639495 Change-Id: Ie09094928d21e8449a372075a5e88a50c628fb04 * Use `SetDimWithStatus` instead of `set_dim`. This would prevent a crash due to overflow. PiperOrigin-RevId: 416083115 Change-Id: Id2a0c11f5940ec8e28e4e39a37f506c875b5aca5 * Use builders for tensor shapes to prevent overflows. PiperOrigin-RevId: 416083152 Change-Id: I7476345ce34039bde8afa657412ff0b7b64b03d9 * Prevent overflow causing crash in `SerializeManySparse` PiperOrigin-RevId: 416083201 Change-Id: Ib5244a92ea4e84a1af5e7f3339e6876a88a533a7 * Add a fix for Check Failure due to integer overflow PiperOrigin-RevId: 416216408 Change-Id: I16d3e2ce35d88b91e8a4ca38b9ebbbf7a05ee792 * Properly validate sparse tensor in `SparseTensorSliceDataset` Existing validation was incomplete. PiperOrigin-RevId: 415375048 Change-Id: I14cd18f29ede73286f3ffac35171bd15828997e9 * Fix check-fail when bincount ops are passed invalid values. PiperOrigin-RevId: 415063028 Change-Id: I20f8dc09933ddca1111c4efbf9a3a1e863215d02 * Fix nullptr exception in QuantizedMaxPool op when empty list is sent to min_input or max_input parameters. PiperOrigin-RevId: 413960973 Change-Id: I9e3ded593f3c4eabf0d6d5dc356e6a19a3ad2682 * Fix Null-pointer dereference in BuildXlaCompilationCache If ConfigProto is not used, then use the default settings which is to allow all devices. PiperOrigin-RevId: 420391800 Change-Id: I88161ad7042990aef678e77b597a2fb2c8f815be * Cleanup and remove duplicate validation in `SparseCount`. We have valdiation that is duplicated, checking different conditions, in different formats and failing to capture all cases. This should fix all the previous bugs. PiperOrigin-RevId: 414886981 Change-Id: Ibf0bba0beb057b76d505324bb9487565daf95f01 * Further validate sparse tensor for `SparseCount`: indices must be valid within dense shape. PiperOrigin-RevId: 414888122 Change-Id: I4552bd74c135ecd4bcb5448acc0a3ce9402d8286 * Prevent crash due to integer overflow followed by allocating negative sized array. PiperOrigin-RevId: 414891322 Change-Id: I5df390e0dc1d9f115209293708950cdf9306931c * Fix crash when importing an invalid graph with an empty op type to TFG Found by proto fuzzing. PiperOrigin-RevId: 413847837 Change-Id: Icac24d1b389c5661800fb4d622dff0b31d846cca * Fix crash when importing an invalid graph with functions with empty names Found by proto fuzzing. PiperOrigin-RevId: 413918956 Change-Id: I2cb898d6561070cfbcf448ea0da5e838438f3e92 * Gracefully catch errors when importing an invalid graphdef in MLIR TFG When importing a "generic function" in TFG, we don't build a Graph in memory and so we need to implement a bit more checking in the importer itself. This particular case catches duplicated names between nodes or between nodes and function arguments, and fixes a crash found by the fuzzer. PiperOrigin-RevId: 409331027 Change-Id: Ibaf6290f67908c020c5103a7e009bdffd88690e2 * [lite] Add check for bias_size is zero to avoid division by zero. This shouldn't happen for properly converted models. Just safety check PiperOrigin-RevId: 416383645 Change-Id: If8e508bf696ae8ecfb927e69c139a8ccf7fe60cb * [lite] Add validation check for dilation height/width to be positive integers. PiperOrigin-RevId: 416429178 Change-Id: If7cdcddca54486434d9b2f06e7e2b401d7c3ee25 * [lite] Update TfLiteIntArrayCreate to return size_t PiperOrigin-RevId: 416439896 Change-Id: I847f69b68d1ddaff4b1e925a09b8b69c1756653b * [lite] Move MultiplyAndCheckOverflow to util to be able to share it. PiperOrigin-RevId: 416897229 Change-Id: I5feb44881bdcbb6ed911da4f17c55bb978754059 * [lite] Check for overflow when creating required bytes. PiperOrigin-RevId: 417629001 Change-Id: Ia7feb3ea8e988f4fd4b3c98c1a1fed4557d99fd7 * Prevent segfault in `embedding_lookup_sparse.cc` Previous fixes missed one additional case. PiperOrigin-RevId: 417676944 Change-Id: I8ab412155cf9b1e897448a6611d209eaa7ca9e66 * [lite] add validation check for sparse fully connected PiperOrigin-RevId: 417629354 Change-Id: If96171c4bd4f5fdb01d6368d6deab19d1c9beca7 * Remove a `DCHECK`-fail, log an error instead. `DCHECK` in debug mode results in crashes. TensorFlow has had multiple vulnerabilities due to this. Outside of debug mode, `DCHECK` is a no-op. A better alternative is to report an error to the log buffer and continue. This should happen both in debug mode and in prod mode. PiperOrigin-RevId: 408375925 Change-Id: Id5b3e19c73f3fbe0cc4bba26ca44ff9607bb6356 * [lite] Add some safety checks to avoid out of bound access for sparsity format PiperOrigin-RevId: 416910386 Change-Id: Ic0b4dc048dc4b5a6309c572b8c4c9f776e4db60a * Prevent a crash due to heap OOB write in grappler. PiperOrigin-RevId: 408318417 Change-Id: If095feb8c001e3a8ac4a85b7387b81e8309df47d * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420335872 Change-Id: I331ec2544a08d3cc3063a74af342cceae655b3dc * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420355705 Change-Id: If437973d0cd7686a221679d4123cb12f79697fe0 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359120 Change-Id: Ifb43401b1fd3e023c685dc3a74b3b655090e1ce6 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359138 Change-Id: I8afc97448b1e730ac5883c2033f3b0e544b8fb58 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359156 Change-Id: I992e93ed8423eef87bfcfc84b0c877131d6f916d * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359212 Change-Id: I172811749d2e7b901399f63df4fd1523447c6682 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359224 Change-Id: I7bfc1df9cf931f45ec85d4878874ef41b9c55474 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359231 Change-Id: If2049dbeb46fb8ff6df7c8e077cee8be3872e5b4 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420359237 Change-Id: I7fa45e888deff612ca53a4f8610cfad8f28e9671 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420360028 Change-Id: Icd8a7ba3e47c2ff63a26a2fe007737ef01c0cb1d * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420360036 Change-Id: I13eb94736af3397261cf0d46214ddb5a2af9d92b * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420363556 Change-Id: I3225120cd6545462174641581a365ead0eb179c3 * Use `tempfile.mkdtemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420369603 Change-Id: I2cf40b13f41cc01000c2c21a483a2d680194dba2 * Use `tempfile.mkdtemp` instead of `tempfile.mktemp` to create directories. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do: just a name change PiperOrigin-RevId: 420370858 Change-Id: I44a0849d161132eacd4f3881fdb615e09c0f02a2 * Use `tempfile.mkstemp` instead of `tempfile.mktemp`. The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html). The switch is easy to do. PiperOrigin-RevId: 420384092 Change-Id: I8721c09ccc4de589b5a45d38e7ebc440160c72b8 * Fix `logging_ops_test` which fails on Windows PiperOrigin-RevId: 420423302 Change-Id: I688a5e77831561c034c3e5bb09bdb0cedea63801 * Fix `debugger_cli_common_test` which breaks Windows PiperOrigin-RevId: 420425157 Change-Id: I29596c2ef7d0dc13e3678c48394d31ab0bfcbd24 * Validate `proto.dtype()` before calling `set_dtype()`. This prevents a `DCHECK`-fail when the proto contains an invalid dtype for a tensor shape with 0 elements or for an incomplete tensor shape. PiperOrigin-RevId: 408369083 Change-Id: Ia21a3e3d62a90d642a4561f08f3b543e5ad00c46 * Prevent null dereference read in `SpecializeType()` For some adversarial protos, the attribute for a key might not exist. PiperOrigin-RevId: 408382090 Change-Id: Ie7eabe532c9ff280fce5dce1f6cdb93c76c2e040 * Properly handle the case where `SpecializeType()` returns an error `Status`. If the error case in `SpecializeType()` is reached, then we would get a crash when trying to access the value of an errorenous `StatusOr` object PiperOrigin-RevId: 408380069 Change-Id: If3c3fc876dcf9384d5ec7a4985adc68c23ea7318 * Fix heap OOB read/write due to incorrect indexing. PiperOrigin-RevId: 408578046 Change-Id: Ifc9ffea49e5890f55fcb2c27568611052c3ddcfa * Prevent copying uninitialized data in `AssignOp`. This prevents harder to debug undefined behaviors that cannot be traced back to the original tensor after assignments occur earlier in the graph execution. Several of these undefined behaviors are just reference bindings to null pointers, which are caught when running under ubsan/asan. PiperOrigin-RevId: 408654780 Change-Id: Iad2ec40d43f5fd7ea016c20283356c12d5ddeab1 * Prevent integer overflow in `OpLevelCostEstimator::CalculateTensorSize`. In order to not change the API, we return a negative value in case of overflow. A better fix is to change the API to return a status instead. PiperOrigin-RevId: 408713061 Change-Id: I3771475b0c72a2844a3854086966562fd33f2da5 * Prevent integer overflow in `OpLevelCostEstimator::CalculateOutputSize`. In order to not change the API, we return a negative value in case of overflow. A better fix is to change the API to return a status instead. PiperOrigin-RevId: 408701427 Change-Id: Idf31e7f0bf18ca824d084fdd355e1f653f145c20 * Prevent integer overflow in `CalculateTensorSize`. In order to not change the API, we return a negative value in case of overflow. A better fix is to change the API to return a status instead. PiperOrigin-RevId: 408714915 Change-Id: I110ec4e1c5bbf4d7ca7ef7c068dfd3e8bc7190cd * Prevent null dereference read in `GetInitOp`. We have a map of maps. We test that the key exists in the first map but then we don't have any validation that this also means the second map has the needed key. In the scenarios where this is not the case, we'll dereference a nullptr, if we don't have this check PiperOrigin-RevId: 408739325 Change-Id: If9bb7ed759aba1f3b56a34913f209508dbaf65ce * Fix memory leak when a graph node is invalid. If a graph node is invalid but a kernel is created then we set the kernel back to `nullptr` but we forget to delete it. Hence, we get a memory leak. PiperOrigin-RevId: 408968108 Change-Id: I1d8a9d0d8988ed5e08be8b9f2004ce1b4cd11b7c * Fix abort caused by allocating a too large vector. We need to make sure that the number of dimensions in a shape is within limits. PiperOrigin-RevId: 408997911 Change-Id: If59e1c23f2ec9c2d4ff4d8632fd62b2a7773a4eb * Eliminate `CHECK`-fails from `IsSimplifiableReshape` via `MakeShape(<invalid shape>)` PiperOrigin-RevId: 409166738 Change-Id: I7f0a3590b8acae3f3e3e2fe636e1f5ef285693cf * Remove `CHECK`-fails from `IsSimplifiableReshape` PiperOrigin-RevId: 409164987 Change-Id: I58c7dd459ff348c3dbae95e00c4c5e63b30a4e65 * Make `IsSimplifiableReshape` return `Status` instead of `bool`. This is to allow remove `CHECK`-fails in subsequent commits. PiperOrigin-RevId: 409160987 Change-Id: I3f050218a3832271395c4372a0b8ea05f1c03d80 * Prevent a null-pointer dereference / `CHECK`-fail in grappler. PiperOrigin-RevId: 409187354 Change-Id: I369c249cca32e6c56ec193f0ebbf2f2768fc7d43 * Use `PartialTensorShape` instead of `TensorShape`. `TensorShape` constructor throws a CHECK-fail if shape is partial/overflows which the other doesn't. We are only determining the number of elements in the shape and partial shape should be used as it returns negative number when needed. PiperOrigin-RevId: 409205384 Change-Id: Ia56542ff9ec758f2c9ffc7e4dcc9fa7eecd86e7b * Validate real and expected type of arguments to cwise ops. Without this validation, it is possible to trigger a `CHECK`-fail denial of service. This is a rollforward of a previous commit which was rolled back as it was relying on RTTI. This time we don't use RTTI, we replace `typeid(Tin).name()` with a double function call, `DataTypeString(DataTypeToEnum<Tin>::v())`. PiperOrigin-RevId: 409340416 Change-Id: I96080b2796729a3a9b65e7c68307ac276070f2f0 * Prevent use after free in `DecodePng` kernel. We are cleaning up the memory in `decode` and then we are using an `OP_REQUIRES` to check an invariant on the `decode` data. PiperOrigin-RevId: 409299145 Change-Id: I4eb93aaca52483eb202e89b78df07fbb2f6cb254 * In case of error in `DecodePng`, end kernel execution immediately. There are scenarios where we detect an error in `DecodePng` kernel, yet the execution continues for a while longer before an error is thrown. This is not safe. PiperOrigin-RevId: 409299935 Change-Id: Ife488b410148032ae777f59bc51864e172553fdf * Prevent memory leak in decoding PNG images. PiperOrigin-RevId: 409300653 Change-Id: I6182124c545989cef80cefd439b659095920763b * Eliminate debug `CHECK`-fail from `function.cc` PiperOrigin-RevId: 409416119 Change-Id: I8376ee464d434e9b970ff0ad49edfdaa2a273cfe * Eliminate `CHECK`-fail from `function.cc`. PiperOrigin-RevId: 409414744 Change-Id: Ic854e12ab2edb88b165d32e2d632c4ee654d71ad * Prevent `CHECK`-fail when building reference tensor. The tensor constructor does not allow reference dtypes, as these should not show up explicitly. However, when passed these invalid types instead of building an invalid object the constructor crashes via a `CHECK`-fail. We have a static builder that properly handles this case but is not applicable given current usage. Instead, before calling the constructor, we can check that the dtype is not a reference type and return an error otherwise, given that the dtype is user controlled so malicious users can trigger denial of service. PiperOrigin-RevId: 409662503 Change-Id: I5892f831fde7f276cd7ab34519cf6b8061c71a59 * Prevent overflow in `CalculateTensorElementCount` Grappler cost estimation sometimes computes the number of elements in a tensor by multiplying all the dimensions in a shape. However, these tensors can also be controlled by users so a malicious attacker can trigger overflow that can be exploited. PiperOrigin-RevId: 409575048 Change-Id: I7a958875ba6f3ad9cb5b9943fe5d459efcbe4557 * Prevent overflow in grappler cost estimation of crop&resize op. The crop parameters are user controlled, so we should make sure a user can not trigger an overflow maliciously. PiperOrigin-RevId: 409670234 Change-Id: I7994734a98b037c5642e051240329d16f959aae4 * Prevent null pointer dereference in `mutable_graph_view` PiperOrigin-RevId: 409684472 Change-Id: I577eb9d9ac470fcec0501423171e739a4ec0cb5c * Prevent null pointer dereference in constant folding. Under certain conditions, an invalid protobuf saved model with invalid nodes would be loaded. During optimization phase, Grappler optimizer will then dereference a null pointer. PiperOrigin-RevId: 409683530 Change-Id: I1f10340a7ec384bc9bc587300390f1078cf5caa0 * Fix `CHECK`-failure caused by constant folding code. We're losing a `const` qualifier here, but unless we get to use more `StatusOr` objects, this is the best alternative. PiperOrigin-RevId: 410072241 Change-Id: I69535c91490f0d23facb9587d2ff59db0782cda6 * Prevent stack overflow when FunctionLib in GraphDef has a self-recursive function. It is likely that no recursivity is supported, but we should handle this separately. PiperOrigin-RevId: 414860329 Change-Id: I02a2270e86282b37362ddd485eeef16fb986a9e0 * Merge pull request tensorflow#52707 from elfringham:init_ops_test_fix PiperOrigin-RevId: 416941851 Change-Id: Iefa5a9b841b053b36f6b105cd82c9d32d5e47850 * Merge pull request tensorflow#53695 from yongtang:53660-tf.sparse.split-crash PiperOrigin-RevId: 420811652 Change-Id: I83742482770ba0bf7c3ccd57508c40fb9cdbe2f7 * Add static factory builders for (partial) tensor shapes. We have them in `TensorShapeBase` but calling them directly from there is cumbersome: ```cc TensorShapeBase<TensorShape>::BuildTensorShape(...) TensorShapeBase<PartialTensorShape>::BuildTensorShape(...) ``` With this change, above lines change to ```cc TensorShape::BuildTensorShape(...) PartialTensorShape::BuildTensorShape(...) ``` This should make it easier to remove `CHECK`-fails from code, preventing breakages similar to multiple CVEs we've had this year. PiperOrigin-RevId: 407922426 Change-Id: I9e12f112457567a46528d170dc76137d553c9e81 * Prevent `CHECK`-fail when decoding resource handles from proto In certain scenarios, the proto might contain tensors that have too many elements (overflow). This is a `CHECK`-fail in general, but we should prevent this, given how many CVEs caused by that we have received this year (a large fraction of 200). PiperOrigin-RevId: 408049766 Change-Id: I2ac20b247aa8ed9110846fbdb7a0a9401f2c168c * Update tensorflow/core/grappler/optimizers/constant_folding.cc * Update tensorflow/core/grappler/optimizers/constant_folding.cc * Update tensorflow/core/grappler/optimizers/constant_folding.cc * Update tensorflow/core/grappler/optimizers/constant_folding.cc * Check for type inference error on node construction. PiperOrigin-RevId: 409415804 Change-Id: Ieb6e020906b96f522bf8e2fa103715ddbbdc434a * Fix crash when an invalid graph (empty handle_data) is provided to the TFG importer Found by the fuzzer. PiperOrigin-RevId: 413036574 Change-Id: Ib90e542c04ccc46416bce3201e23f339e5bd1097 * Fix crash when importing invalid graph (attribute with empty name) to TFG Found by the fuzzer PiperOrigin-RevId: 413037608 Change-Id: I7220932e261bd71848d8bd6d0ee654e83c7fa2e3 * Fix crash when importing invalid graph (func attribute with empty name) to TFG Found by the fuzzer PiperOrigin-RevId: 413200989 Change-Id: Ib81abccaf33f05a69c780c937710699f761ede72 * Fix crash on invalid graph importing to TFG when there is a func_attr with an empty key. Found by fuzzing. PiperOrigin-RevId: 414089748 Change-Id: Ibdbb5aed29f36abd4bb66f7ef854868dc6b9d95c * lite: Update Windows tensorflowlite_flex.dll build Removed "windows_export_all_symbols" feature since Flex delegate only requires to expose `TF_AcquireFlexDelegate` symbol. This change is needed for Issue#43367. PiperOrigin-RevId: 424060071 Change-Id: I8874ce6b107f6db9c5445b65e55073ea46266c76 * Fix `OP_REQUIRES` cherry-pick error * Bump the maximum threshold before erroring PiperOrigin-RevId: 424653571 Change-Id: Ic2d9f3a7db627d78cde80ad415105f3d53735b3b * Disable broken tests * Insert release notes place-fill * Update version numbers to 2.7.1 * Bump ICU to 69.1 to handle CVE-2020-10531. See [CVE-2020-10531](https://nvd.nist.gov/vuln/detail/CVE-2020-10531), it's an integer overflow. PiperOrigin-RevId: 424908014 Change-Id: Ib8d6f8b527fb1fd9d887eaf487afc17d42333c8a * Revert "Bump ICU to 69.1 to handle CVE-2020-10531." * Revert "Revert "Bump ICU to 69.1 to handle CVE-2020-10531."" * Update third_party/icu/workspace.bzl * Update third_party/icu/workspace.bzl * Update RELEASE.md * Set Env Variable to override Setuptools new behavior PiperOrigin-RevId: 423468055 Change-Id: I5b148103e1372a5eb73570bc77face27dbd5f914 * Reorder tags to fix buildifier linting * Disable flaky test Co-authored-by: Duncan Riach <duncan@nvidia.com> Co-authored-by: Frederic Bastien <fbastien@nvidia.com> Co-authored-by: Mihai Maruseac <mihaimaruseac@google.com> Co-authored-by: Terry Heo <terryheo@google.com> Co-authored-by: Karim Nosir <karimnosseir@google.com> Co-authored-by: Katherine Wu <kathywu@google.com> Co-authored-by: Isha Arkatkar <ishark@google.com> Co-authored-by: Andrew Audibert <aaudibert@google.com> Co-authored-by: A. Unique TensorFlower <gardener@tensorflow.org> Co-authored-by: Edward Loper <edloper@google.com> Co-authored-by: Smit Hinsu <hinsu@google.com> Co-authored-by: Mehdi Amini <aminim@google.com> Co-authored-by: Mihai Maruseac <mihai.maruseac@gmail.com> Co-authored-by: Dan Moldovan <mdan@google.com> Co-authored-by: TensorFlow Release Automation <jenkins@tensorflow.org>

google-ml-butler bot added the size:S CL Change Size: Small label Oct 27, 2021

google-ml-butler bot assigned gbaned Oct 27, 2021

google-cla bot added the cla: yes label Oct 27, 2021

gbaned added comp:core issues related to core part of tensorflow prtype:bugfix PR to fix a bug labels Oct 28, 2021

gbaned added this to Assigned Reviewer in PR Queue via automation Oct 28, 2021

gbaned requested a review from rohan100jain October 28, 2021 04:20

google-ml-butler bot added the awaiting review Pull request awaiting review label Oct 28, 2021

elfringham force-pushed the init_ops_test_fix branch from 8aa4225 to 8a363d6 Compare November 2, 2021 13:32

gbaned requested a review from jpienaar December 3, 2021 15:35

Test for case that would lead to undefined behaviour and throw error …

55deb9b

…if found

elfringham force-pushed the init_ops_test_fix branch from 8a363d6 to 55deb9b Compare December 8, 2021 15:40

penpornk approved these changes Dec 16, 2021

View reviewed changes

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Dec 16, 2021

PR Queue automation moved this from Assigned Reviewer to Approved by Reviewer Dec 16, 2021

penpornk removed the awaiting review Pull request awaiting review label Dec 16, 2021

kokoro-team removed the kokoro:force-run Tests on submitted change label Dec 16, 2021

mihaimaruseac assigned mihaimaruseac and unassigned gbaned Dec 16, 2021

copybara-service bot merged commit f014775 into tensorflow:master Dec 17, 2021

google-ml-butler bot removed the ready to pull PR ready for merge process label Dec 17, 2021

mihaimaruseac pushed a commit that referenced this pull request Jan 25, 2022

Merge pull request #52707 from elfringham:init_ops_test_fix

5e2582d

PiperOrigin-RevId: 416941851 Change-Id: Iefa5a9b841b053b36f6b105cd82c9d32d5e47850

mihaimaruseac pushed a commit that referenced this pull request Jan 25, 2022

Merge pull request #52707 from elfringham:init_ops_test_fix

0539b69

PiperOrigin-RevId: 416941851 Change-Id: Iefa5a9b841b053b36f6b105cd82c9d32d5e47850

mihaimaruseac pushed a commit that referenced this pull request Jan 25, 2022

Merge pull request #52707 from elfringham:init_ops_test_fix

e896767

PiperOrigin-RevId: 416941851 Change-Id: Iefa5a9b841b053b36f6b105cd82c9d32d5e47850

mihaimaruseac added a commit that referenced this pull request Jan 25, 2022

Merge pull request #54070 from tensorflow/cherrypick-52707-on-r2.7

fa50f42

Merge pull request #52707 from elfringham:init_ops_test_fix

mihaimaruseac added a commit that referenced this pull request Jan 25, 2022

Merge pull request #54071 from tensorflow/cherrypick-5e2582d998b-on-r2.6

727391e

Merge pull request #52707 from elfringham:init_ops_test_fix

mihaimaruseac added a commit that referenced this pull request Jan 25, 2022

Merge pull request #54072 from tensorflow/cherrypick-e896767551a-on-r2.5

b44d9de

Merge pull request #52707 from elfringham:init_ops_test_fix

elfringham deleted the init_ops_test_fix branch January 26, 2023 14:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test for case of undefined behaviour and throw error if found #52707

Test for case of undefined behaviour and throw error if found #52707

elfringham commented Oct 27, 2021

penpornk left a comment

Test for case of undefined behaviour and throw error if found #52707

Test for case of undefined behaviour and throw error if found #52707

Conversation

elfringham commented Oct 27, 2021

penpornk left a comment

Choose a reason for hiding this comment