Skip to content

Conversation

@marstr
Copy link
Contributor

@marstr marstr commented May 16, 2019

Closes #1

@marstr marstr requested review from abhidnya13 and rayluo May 16, 2019 16:40
Copy link
Contributor

@rayluo rayluo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Despite the seemingly lengthy comments below, I really want to thank you for your trailblazing work in this part! This was an area that we haven't previously explored, and you made it happen!

pb_data = self.pbData
buffer = ctypes.c_buffer(cb_data)
_memcpy(buffer, pb_data, cb_data)
_local_free(pb_data)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this LocalFree() pattern also showed up in that stackoverflow question which we modeled from. Out of curiosity, is such free() here really necessary/desirable? The current way seems making raw() unpredictable when being accidentally called twice?

Copy link
Contributor Author

@marstr marstr May 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the _local_free is definitely necessary as we don't want to create a memory leak! However, you're totally right that it should be safe to call this method twice in a way it just isn't right now. I'll fix that up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b9becbc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I'm going to have to walk this change back. I got to a Windows machine and ran the tests, and curiously pulling this object out into the destructor causes a seg-fault that crashes the host Python process with it.

Copy link
Contributor

@rayluo rayluo May 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that is the case, can we have one more try, pun not intended, to move that _local_free() out of raw(), and put it into protect() in this manner? On paper, it should work.

    def protect(self, ...):
        ...
        try:
            return result.raw()
        finally:
            _local_free(result.pbData)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it :) I'll give it another whack in the morning.

Copy link
Contributor Author

@marstr marstr May 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the trick ended up being that LocalFree MUST NOT be called memory allocated by ctypes.create_string_buffer, but MUST be called on memory allocated by the CryptProtectData and CryptUnprotectData functions. It just so happened that I only ever called raw on the latter, and never called raw for the former. This means that using try & finally allows us to manage the lifetime of only specific structs, where as moving it into a __del__ method was breaking the contract.

Long story short, this is fixed in 161e6cf 2a25b5f. On the otherhand, a4efa65 80e735c can be ignored.

edit: Fixing commit-id's after rebase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing all the hard work to validate our latest fix, and documeting your finding! For the record, even if that 80e735c would work, I would still prefer the 2a25b5f approach. Because that's fundamentally the right pattern to manage memory (came from my 7 years on C/C++).

🚢

@marstr marstr requested a review from rayluo May 21, 2019 19:19
Copy link
Contributor

@rayluo rayluo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for relentlessly addressing feedbacks and driving the code to a higher quality!

@marstr marstr merged commit b7c303f into AzureAD:dev May 22, 2019
@marstr marstr deleted the dp_api branch May 22, 2019 18:48
@rayluo rayluo mentioned this pull request Jul 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a DP API Token Cache (Windows Only)

2 participants