[Foundation] Fix random memory access / memory corruption in NSFastEnumerator. Fixes #xamarin/maccore@1347. #16920
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…umerator. Fixes #xamarin/maccore@1347. The NSFastEnumerator protocol in Objective-C works like this: 1. A selector is called on the collection in question, where we give the native method a pointer to a state structure. Upon return, the state structure will have two important pointers: one to a C array with the pointers to enumerate, and another pointer that points to a value determining whether the collection was modified since enumeration started. 2. In the original managed implementation, we'd store the state (a struct) as an instance field of NSFastEnumerator. This works fine... most of the time. Unfortunately most of the time isn't good enough, because this may happen: a. The native iterator function might store a pointer into the state structure itself as the pointer to check for a modified collection. b. The GC runs, and moves memory around. Now suddenly the pointer we have to read to check if the collection was modified is pointing to the previous location of the state structure, which could be anything or anywhere, and the app subsequently tries to summon Murphy from his pantheon in the sky (admittedly with limited success). The fix is to use native memory (which the GC won't move around) to store the enumeration state. Also optimize memory usage a little bit by allocating one big blob for both the state and the array of pointers we pass to the native iterator method. Fixes xamarin/maccore#1347. Fixes xamarin/maccore#2606.
This comment was marked as outdated.
This comment was marked as outdated.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mandel-macaque
approved these changes
Dec 1, 2022
✅ API diff for current PR / commitLegacy Xamarin (No breaking changes)
NET (empty diffs)
❗ API diff vs stable (Breaking changes)Legacy Xamarin (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:).NET (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)✅ Generator diffGenerator diff is empty Pipeline on Agent |
💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻✅ All tests on macOS M1 - Mac Big Sur (11.5) passed. Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
🚀 [CI Build] Test results 🚀Test results✅ All tests passed on VSTS: simulator tests. 🎉 All 223 tests passed 🎉 Tests counts✅ bcl: All 69 tests passed. Html Report (VSDrops) Download Pipeline on Agent |
|
/sudo backport xcode14.1 |
|
/sudo backport release/6.0.4xx-xcode14.1 |
|
/sudo backport release/7.0.1xx |
|
I couldn't create a backport to xcode14.1 for you. :( Please check the Action logs for more details. |
|
I couldn't create a backport to release/6.0.4xx-xcode14.1 for you. :( Please check the Action logs for more details. |
|
I couldn't create a backport to release/7.0.1xx for you. :( Please check the Action logs for more details. |
This was referenced Dec 2, 2022
rolfbjarne
added a commit
that referenced
this pull request
Dec 6, 2022
…emory corruption in NSFastEnumerator. Fixes #xamarin/maccore@1347. (#16943) The NSFastEnumerator protocol in Objective-C works like this: 1. A selector is called on the collection in question, where we give the native method a pointer to a state structure. Upon return, the state structure will have two important pointers: one to a C array with the pointers to enumerate, and another pointer that points to a value determining whether the collection was modified since enumeration started. 2. In the original managed implementation, we'd store the state (a struct) as an instance field of NSFastEnumerator. This works fine... most of the time. Unfortunately most of the time isn't good enough, because this may happen: a. The native iterator function might store a pointer into the state structure itself as the pointer to check for a modified collection. b. The GC runs, and moves memory around. Now suddenly the pointer we have to read to check if the collection was modified is pointing to the previous location of the state structure, which could be anything or anywhere, and the app subsequently tries to summon Murphy from his pantheon in the sky (admittedly with limited success). The fix is to use native memory (which the GC won't move around) to store the enumeration state. Also optimize memory usage a little bit by allocating one big blob for both the state and the array of pointers we pass to the native iterator method. Fixes xamarin/maccore#1347. Fixes xamarin/maccore#2606. Backport of #16920.
rolfbjarne
added a commit
that referenced
this pull request
Dec 6, 2022
…uption in NSFastEnumerator. Fixes #xamarin/maccore@1347. (#16942) The NSFastEnumerator protocol in Objective-C works like this: 1. A selector is called on the collection in question, where we give the native method a pointer to a state structure. Upon return, the state structure will have two important pointers: one to a C array with the pointers to enumerate, and another pointer that points to a value determining whether the collection was modified since enumeration started. 2. In the original managed implementation, we'd store the state (a struct) as an instance field of NSFastEnumerator. This works fine... most of the time. Unfortunately most of the time isn't good enough, because this may happen: a. The native iterator function might store a pointer into the state structure itself as the pointer to check for a modified collection. b. The GC runs, and moves memory around. Now suddenly the pointer we have to read to check if the collection was modified is pointing to the previous location of the state structure, which could be anything or anywhere, and the app subsequently tries to summon Murphy from his pantheon in the sky (admittedly with limited success). The fix is to use native memory (which the GC won't move around) to store the enumeration state. Also optimize memory usage a little bit by allocating one big blob for both the state and the array of pointers we pass to the native iterator method. Fixes xamarin/maccore#1347. Fixes xamarin/maccore#2606. Backport of #16920.
rolfbjarne
added a commit
that referenced
this pull request
Dec 6, 2022
… in NSFastEnumerator. Fixes #xamarin/maccore@1347. (#16941) The NSFastEnumerator protocol in Objective-C works like this: 1. A selector is called on the collection in question, where we give the native method a pointer to a state structure. Upon return, the state structure will have two important pointers: one to a C array with the pointers to enumerate, and another pointer that points to a value determining whether the collection was modified since enumeration started. 2. In the original managed implementation, we'd store the state (a struct) as an instance field of NSFastEnumerator. This works fine... most of the time. Unfortunately most of the time isn't good enough, because this may happen: a. The native iterator function might store a pointer into the state structure itself as the pointer to check for a modified collection. b. The GC runs, and moves memory around. Now suddenly the pointer we have to read to check if the collection was modified is pointing to the previous location of the state structure, which could be anything or anywhere, and the app subsequently tries to summon Murphy from his pantheon in the sky (admittedly with limited success). The fix is to use native memory (which the GC won't move around) to store the enumeration state. Also optimize memory usage a little bit by allocating one big blob for both the state and the array of pointers we pass to the native iterator method. Fixes xamarin/maccore#1347. Fixes xamarin/maccore#2606. Backport of #16920
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The NSFastEnumerator protocol in Objective-C works like this:
native method a pointer to a state structure. Upon return, the state
structure will have two important pointers: one to a C array with the
pointers to enumerate, and another pointer that points to a value
determining whether the collection was modified since enumeration started.
an instance field of NSFastEnumerator.
This works fine... most of the time. Unfortunately most of the time isn't good
enough, because this may happen:
a. The native iterator function might store a pointer into the state structure
itself as the pointer to check for a modified collection.
b. The GC runs, and moves memory around.
Now suddenly the pointer we have to read to check if the collection was
modified is pointing to the previous location of the state structure, which
could be anything or anywhere, and the app subsequently tries to summon Murphy
from his pantheon in the sky (admittedly with limited success).
The fix is to use native memory (which the GC won't move around) to store the
enumeration state.
Also optimize memory usage a little bit by allocating one big blob for both
the state and the array of pointers we pass to the native iterator method.
Fixes https://github.com/xamarin/maccore/issues/1347.
Fixes https://github.com/xamarin/maccore/issues/2606.