Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix attribute comparison for ProxyGenerator cache #219

Merged

Conversation

thomaslevesque
Copy link
Contributor

Fixes #77

At this point, this PR is mostly a proof of concept; I didn't try to preserve the existing API for backwards compatibility, I just changed the type of AdditionalAttributes from CustomAttributeBuilder to a new CustomAttributeInfo class. This class has the same constructors as CustomAttributeBuilder, and overrides Equals and GetHashCode. I also introduce a CustomAttributeInfo.FromExpression method to make it easier to define attributes:

var info = CustomAttributeInfo.FromExpression(
    () => new MyAttribute("foo") { Value = 42, Type = typeof(int) });
options.AdditionalAttributes.Add(info);

@jonorossi jonorossi added this to the v4.0 milestone Oct 20, 2016
@jonorossi
Copy link
Member

@thomaslevesque thanks, it looks good, sorry for the slow response to review this. I do like FromExpression, much better than the now obsolete code that pulls apart an actual Attribute instance to try rebuilding it.

Now to discuss whether this should be a breaking change. Looking at the 3 main mocking libraries only FakeItEasy uses AdditionalAttributes. You guys expose CustomAttributeBuilder via
IFakeOptions.WithAdditionalAttributes, are you happy for that to be a breaking change to FakeItEasy and are you comfortable exposing a DynamicProxy type on your IFakeOptions interface? If you are ILMerging you'll also have to expose that type.

CustomAttributeInfo works, however this all makes me wonder whether we should just side step this and inspect CustomAttributeBuilder.m_con (ctor on mono) and CustomAttributeBuilder.m_blob (data on mono) using reflection. It'll be a faster comparison, however the IL blob will preserve property order from the code which is probably want we want here anyway. We already use reflection to access many public members of the BCL types. Both options aren't ideal, would love to hear your (and anyone else's) thoughts.

@thomaslevesque
Copy link
Contributor Author

are you happy for that to be a breaking change to FakeItEasy

I don't mind the breaking change. It's not a very widely used feature, so it shouldn't affect many users, and the fix will be pretty easy anyway. @adamralph, @blairconrad, what do you think?

and are you comfortable exposing a DynamicProxy type on your IFakeOptions interface? If you are ILMerging you'll also have to expose that type.

That's a good question, I hadn't thought about that. We already expose part of the DynamicProxy API as a result of using ILMerge, but not directly as part of the FakeItEasy API. I'd rather avoid that if possible, but I'm not sure how. I guess we could create yet another CustomAttributeBuilder-like type that would be specific to FakeItEasy, but it's not a very appealing option. Or we could expose only a method that accepts an Expression<Func<Attribute>>.

CustomAttributeInfo works, however this all makes me wonder whether we should just side step this and inspect CustomAttributeBuilder.m_con (ctor on mono) and CustomAttributeBuilder.m_blob (data on mono) using reflection. It'll be a faster comparison, however the IL blob will preserve property order from the code which is probably want we want here anyway. We already use reflection to access many public members of the BCL types. Both options aren't ideal, would love to hear your (and anyone else's) thoughts.

I guess that would work too, and it would have the benefit of avoiding the breaking change entirely. Of course, as always when using reflection on private members, there's the risk of the internals being changed in a BCL update, but I don't think it's very likely.

@thomaslevesque
Copy link
Contributor Author

Slightly off-topic, but related: while working on this I noticed the obsolete AttributesToAddToGeneratedTypes property. Do you intend to remove it in 4.0?

@blairconrad
Copy link
Contributor

I haven't looked at the changes in depth, but in general it looks good to me.
I love FromExpression (surprised no test coverage for it, though).

I don't mind the breaking change. It's not a very widely used feature, so it shouldn't affect many users, and the fix will be pretty easy anyway. @adamralph, @blairconrad, what do you think?

Well worth it, in my opinion. I'd go for the breaking change.

Or we could expose only a method that accepts an Expression<Func<Attribute>>

Yes! This. 😻

return false;
}
return true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returns false if x has members not in y, but not the other way 'round, no?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most of the comparisons can be short-circuited by checking the length of all the collections before even checking any contents, that would fix this problem.

@jonorossi
Copy link
Member

Slightly off-topic, but related: while working on this I noticed the obsolete AttributesToAddToGeneratedTypes property. Do you intend to remove it in 4.0?

Yep, it was marked obsolete in 2009 so it can go, I actually wrote about it in my last comment then removed the mention to avoid making things more complicated for now. If it gets in your way then remove it, otherwise lets do it separate and update our changelog.

I guess that would work too, and it would have the benefit of avoiding the breaking change entirely. Of course, as always when using reflection on private members, there's the risk of the internals being changed in a BCL update, but I don't think it's very likely.

We can add a FromExpression method either way, the remaining question is do we add CustomAttributeInfo or inspect the privates of CustomAttributeBuilder. I agree that the privates are unlikely to change (however different between the CLR and Mono) which is why I would accept the risk, however after another think/look at DP there is no precedent for accessing privates that are not DP's own or generated by DP (so under its control). I'm heavily leaning towards your CustomAttributeInfo to avoid adding tech debt that someone else has to sort out in the future, and we are about to release a major version anyway so it's a perfect time.

I think we should turn your PoC into a PR ready to merge. i.e. some unit tests, update the changelog and docs.

@jonorossi
Copy link
Member

If it gets in your way then remove it, otherwise lets do it separate and update our changelog.

Actually, let's remove AttributesToAddToGeneratedTypes in this PR too, two birds one stone thingy.

@thomaslevesque
Copy link
Contributor Author

Thanks for the feedback guys! I'll make the requested changes.

I love FromExpression (surprised no test coverage for it, though).

I'll add some tests, but I didn't want to waste time doing it until the idea was approved 😉

@blairconrad
Copy link
Contributor

We can add a FromExpression method either way,

I was also thinking that we could do the same for FakeItEasy even before this PR makes it into a prime-time Castle.Core. It would mean duplicate work that would later be ripped out, but could provide a gentler introduction to the new way to specify attributes. Our clients could have a chance to convert before we make a breaking change.

@jonorossi
Copy link
Member

I was also thinking that we could do the same for FakeItEasy even before this PR makes it into a prime-time Castle.Core.

Up to you, however I plan to release 4.0.0-beta2 after we get this merged. Need to sort out #200, then plan to ship a final release very soon.

@thomaslevesque
Copy link
Contributor Author

I realized my implementation was incomplete, as it didn't properly support arrays as attribute arguments (attributes support primitive types, Object, Type, enums and single dimensional arrays of those types, see §17.1.3 in the C# specs). I updated the code to support this.

I also removed AttributesToAddToGeneratedTypes and related code (e.g. AttributeDisassembler seemed to exist only for this scenario, so I removed it as well).

I still need to add unit tests, but that's all for today.

BTW I'm having a hard time working on the Castle Core code base. The non-.NET Core solution doesn't build in Visual Studio 2015 because of the project.json files. The .NET Core solution builds, but running the tests in the IDE doesn't work (tried with the VS runner and the R# runner). Does it work for you @jonorossi ?

@jonorossi
Copy link
Member

BTW I'm having a hard time working on the Castle Core code base. The non-.NET Core solution doesn't build in Visual Studio 2015 because of the project.json files. The .NET Core solution builds, but running the tests in the IDE doesn't work (tried with the VS runner and the R# runner).

I can't find the .NET CLI issue right now but the updated VS2015 MSBuild targets files break the csproj projects when there is a project.json file in the same directory, so I've been using VS2013, I was actually using VS2013 until someone else reported the problem a while back. My suggestion either use VS2013 or temporarily delete project.json, it is all short lived so I'm not interested in hacking our csproj files to workaround the defect. I use ReSharper's test runner in VS2013 for the .NET Framework projects, and the VS test runner in VS2015 for the .NET Core project.

@adamralph
Copy link

Did we make a decision about the FIE API? I don't think we should expose a Castle.Core type in our public API. We don't currently, and effectively Castle.Core is a hidden implementation detail. We exclude the three types from internalisation when IL merging, but that's only due to the mechanics of Castle.Core. We don't expose those types in our API.

@jonorossi
Copy link
Member

Did we make a decision about the FIE API?

Obviously that is for you guys to ultimately decide, but I liked @thomaslevesque suggestion and that was enough of a solution for me to proceed knowing you guys can continue to keep DynamicProxy an implementation detail:

Or we could expose only a method that accepts an Expression<Func<Attribute>>.

@blairconrad
Copy link
Contributor

@adamralph, I would also like to

expose only a method that accepts an Expression<Func<Attribute>>.

@thomaslevesque
Copy link
Contributor Author

My suggestion either use VS2013 or temporarily delete project.json

Thanks for the suggestion. I no longer have VS2013, so I've been deleting the project.json and project.lock.json.

Did we make a decision about the FIE API?

Not a formal decision, but I think we all agree about not exposing a Castle Core type in our API. My preference would be to just expose the expression-based API, i.e.:

A.Fake<IFoo>(o => o.WithAdditionalAttribute(() => new MyAttribute(42, "blah")));

private static readonly IEqualityComparer<object> ValueComparer = new AttributeArgumentValueEqualityComparer();

private readonly CustomAttributeBuilder builder;
private readonly ConstructorInfo con;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A minor thing, but I don't love "con" as a name. Perhaps its well known, it did slow me down a little. Consider "constructor"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love it either, but I chose it because it's the name used in the CustomAttributeBuilder constructor, so it seemed the natural choice since CustomAttributeInfo is meant to be a "clone" of that class. But I can change it to something else of course.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I say, a minor thing. I'm content to go along.

var assignment = binding as MemberAssignment;
if (assignment == null)
{
throw new ArgumentException("Only assignments bindings are supported");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"assignment bindings"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops. I'll fix it, thanks

return array;
}
}
throw new ArgumentException("Only constant and single-dimensional array expressions are supported");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may reach this line in a case where a single-dimensional array expression is not supported. (When allowArray is false.)
I'd consider reworking the flow to avoid a potentially misleading exception message.
While I'm commenting on it, if such a rework avoided a boolean parameter, I'd consider it an improvement. I never know what these mean. Way back on line 113, I was all '"true"? What's "true" mean?".
But it's not such a huge deal, given that it's a private method and we can find the answer pretty quickly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I thought I had sent this comment, but apparently Github thought I had just started a review... weird)

We may reach this line in a case where a single-dimensional array expression is not supported. (When allowArray is false.)
I'd consider reworking the flow to avoid a potentially misleading exception message.

I thought about it, but in fact, it can only happen if we find an array inside an array. Attribute arguments support single-dimension arrays, but not nested arrays. So in the end the exception message would still be correct.

While I'm commenting on it, if such a rework avoided a boolean parameter, I'd consider it an improvement. I never know what these mean. Way back on line 113, I was all '"true"? What's "true" mean?".

In this case true means that the value is allowed to be an array. The only case where it's false is when called recursively examining array elements (to avoid nested arrays). I can rename it and/or use a named argument on the call site to make things clearer if necessary. I don't see an elegant way to avoid the bool parameter and avoid awkward duplication...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can only happen if we find an array inside an array

Ah! Yes. Thanks. I see it now after rereading. And having you tell me. 😊

I can rename it and/or use a named argument on the call site to make things clearer if necessary.

The meaning of the parameter is quite clear on the inside of the method. And I like the name.
For me, a named argument would improve the call sites. I know they're not everyone's cup of tea, though.

An alternative would be a microtype (probably an enum) so we could have explicitly named values at the call sites, but again, maybe that's too much.

In the end, it's a private method, and not hard for anyone who's confused by the calls to look up the meaning of the bool. Thanks for listening.

{
}

public static CustomAttributeInfo FromExpression(Expression<Func<Attribute>> expression)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure this is perfectly correct, but it would be nice to see it exercised at least once in the tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will. I'm not done yet ;)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry. I must've missed that note. I thought you'd finished it up. What a nag you must think me. Of course now I see the [WIP], which has never left.

@thomaslevesque
Copy link
Contributor Author

OK, I think I'm done now. Added some tests fixed a few issues, and rebased onto master.

@thomaslevesque thomaslevesque changed the title [WIP] Fix attribute comparison for ProxyGenerator cache Fix attribute comparison for ProxyGenerator cache Oct 22, 2016
@blairconrad
Copy link
Contributor

FWIW, it looks very good to me. Thanks, @thomaslevesque!

[Obsolete(
"This property is obsolete and will be removed in future versions. Use AdditionalAttributes property instead. " +
"You can use AttributeUtil class to simplify creating CustomAttributeBuilder instances for common cases.")]
public IList<Attribute> AttributesToAddToGeneratedTypes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please update the changelog with a breaking change for two of these properties and removing IAttributeDisassembler.

@@ -167,6 +167,7 @@
<Compile Include="Components.DictionaryAdapter.Tests\IPhoneWithFetch.cs" />
<Compile Include="Components.DictionaryAdapter.Tests\NameValueCollectionAdapterTests.cs" />
<Compile Include="DynamicProxy.Tests\AttributesToAvoidReplicatingTestCase.cs" />
<Compile Include="CustomAttributeInfoTestCase.cs" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move these unit tests into the DynamicProxy.Tests directory.

}
}
}
;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a stray semicolon at the end here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, don't know how it ended up here...

this.constructorArgs = constructorArgs;

this.properties = namedProperties
.Zip(propertyValues, (p, v) => new { p.Name, Value = v })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enumerable.Zip isn't available on .NET 3.5, it was added in .NET 4.0.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh, forgot about that... I guess I'll just use a for loop. I could also add a compatibility shim, but that would probably be overkill for that use case.

this.builder = new CustomAttributeBuilder(constructor, constructorArgs, namedProperties, propertyValues, namedFields, fieldValues);

this.constructor = constructor;
this.constructorArgs = constructorArgs;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we copy the constructorArgs array to make sure this type stays immutable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Should probably do it for the other parameters too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably do it for the other parameters too.

Scratch that. The other parameters are used to build dictionaries, not kept as is.

Copy link
Contributor

@blairconrad blairconrad Oct 23, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my thought at first as well. Then I had a second thought. It'll probably just reveal my inadequate grasp of what's going on here, but may not individual elements of constructorArgs or the other *Values parameters be single-dimensional arrays, which could be mutated after the fact?

@thomaslevesque
Copy link
Contributor Author

@jonorossi I made the requested changes

@thomaslevesque
Copy link
Contributor Author

btw @jonorossi let me know if you want me to squash some or all the commits before the merge. I don't know how you usually handle this in Castle.Core

@thomaslevesque
Copy link
Contributor Author

may not the other parameters (or even individual args of constructorArgs) be single-dimensional arrays, which could be mutated after the fact?

It doesn't matter for namedProperties, propertyValues, namedFields, and fieldValues, because we don't store them; we just create dictionaries from them.

However you have a point about individual args of constructorArgs, or even values of array properties or fields. But frankly, at this point, a user who is changing the content of an array passed to CustomAttributeInfo is just looking for trouble... so I'm tempted to leave it alone.

@blairconrad
Copy link
Contributor

we just create dictionaries from them

Which is a way to store them, no? In dictionaries!

dict.Add(members[i].Name, values[i]);

shoves the element of the propertiesValues or fieldValues right in the dictionary, for later equality-checking. If the element is an array, it can be mutated.

I do agree that anyone changing the hypothetical arrays after the fact is looking for trouble. Then again, when they find it, they'll probably come back with questions and complaints! 😉
Thanks for considering my comment.

@thomaslevesque
Copy link
Contributor Author

thomaslevesque commented Oct 23, 2016

shoves the element of the propertiesValues or fieldValues right in the dictionary, for later equality-checking. If the element is an array, it can be mutated.

Ah, I thought you were talking about propertiesValues and fieldValues themselves , not the arrays they might contain...

I do agree that anyone changing the hypothetical arrays after the fact is looking for trouble. Then again, when they find it, they'll probably come back with questions and complaints! 😉

True. @jonorossi, what do you think? Should the code account for that by deep-cloning the arrays? (actually, only nested arrays need to be cloned, since all other types allowed as attribute arguments are immutable)

@jonorossi
Copy link
Member

Should the code account for that by deep-cloning the arrays? (actually, only nested arrays need to be cloned, since all other types allowed as attribute arguments are immutable)

CustomAttributeBuilder copies the ctor args (because it holds on to them for some reason) but doesn't clone any constructor argument that is an array so therefore CustomAttributeBuilder also isn't fully immutable (however m_blob is). It doesn't copy any arrays in property/field values because they are written into the byte array by the end of the constructor call and no reference is stored.

To avoid wasted allocations I think we should leave the code as is, and just add an XML comment to the class indicating that any arrays passed to the class are now owned by that class and should not be modified. I agree this is an edge case and I'm happy to direct defect reports to an XML comment.

@jonorossi
Copy link
Member

let me know if you want me to squash some or all the commits before the merge. I don't know how you usually handle this in Castle.Core

I do usually like history in PRs squished if they aren't commits that align to nice clean patches (I'm old school, I like clean patch sets 😉 ), but have got soft over the years as just getting a complete PR seems to be a stretch lately. Many thanks for the work here and being so responsive.

@thomaslevesque
Copy link
Contributor Author

I don't mind squashing into clean commits, that's what we usually do in FakeItEasy.

I squashed the changes down to 4 commits.

@jonorossi
Copy link
Member

@thomaslevesque just wanted to make sure you didn't miss my reply to your question. Were you going to add that XML comment?

@thomaslevesque
Copy link
Contributor Author

@thomaslevesque just wanted to make sure you didn't miss my reply to your question. Were you going to add that XML comment?

Actually, I did miss it... I'll add the comment.

@thomaslevesque
Copy link
Contributor Author

I added the XML comment in a fixup commit; I'll squash it after you approve it.

@jonorossi
Copy link
Member

I added the XML comment in a fixup commit; I'll squash it after you approve it.

Exactly what I was expecting, just without the spelling mistake 😉 :

Arrays passed to this class as constructor arguments or property of field values become owned by this class.

@thomaslevesque
Copy link
Contributor Author

Exactly what I was expecting, just without the spelling mistake 😉 :

Oops! Fixed and squashed

@jonorossi
Copy link
Member

Many thanks @thomaslevesque, I'm ready to merge.

@jonorossi jonorossi merged commit 8fdcf6a into castleproject:master Oct 27, 2016
@blairconrad
Copy link
Contributor

Thanks, @thomaslevesque and @jonorossi!

@thomaslevesque thomaslevesque deleted the fix-attribute-comparison branch October 27, 2016 08:57
@thomaslevesque
Copy link
Contributor Author

Thanks for the merge!

gonchenko added a commit to gonchenko/rhino-mocks that referenced this pull request Jan 9, 2021
* Fix compilation error
> MockRepository.cs(213, 17): [CS0618] 'ProxyGenerationOptions.AttributesToAddToGeneratedTypes' is obsolete: 'This property is obsolete and will be removed in future versions. Use AdditionalAttributes property instead. You can use AttributeUtil class to simplify creating CustomAttributeBuilder instances for common cases.'

See details:
* https://github.com/castleproject/Core/blob/master/CHANGELOG.md#400-beta002-2016-10-28
* castleproject/Core#219
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants