New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable mocking static methods in Mockito #1013

Open
mockitoguy opened this Issue Mar 31, 2017 · 60 comments

Comments

Projects
None yet
@mockitoguy
Member

mockitoguy commented Mar 31, 2017

Static methods mocking with Mockito

This is a placeholder ticket for enabling mocking static methods in Mockito. Example action items that are totally negotiable and can ran in parallel. We're looking for someone who can lead this effort.

  • Research + discuss whether it is a good idea to enable static methods mocking in Mockito. The theory is that it is useful for legacy code, which is most code in the world.
  • Research on how other mocking frameworks do that and whether it is considered useful feature for their users.
  • Design and present for discussion an API for static mocking (slightly relevant ticket: #643)
  • Work with @raphw / ByteBuddy to come up with hacky prototype (the hackier, the better!)
  • Mold the prototype with the API, remove enough rough edges so that the feature is good enough for incubating rollout
  • SHIP IT!
@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Mar 31, 2017

Contributor

I am torn on this one. If we provide this possibility, we might invite developers into bad practices. Every time a developer uses this feature, it should be very explicit that it is bad practice imo. Would it be possible to come up with a solution that is clear on this front, but still allows the developer to opt-in?

Contributor

TimvdLippe commented Mar 31, 2017

I am torn on this one. If we provide this possibility, we might invite developers into bad practices. Every time a developer uses this feature, it should be very explicit that it is bad practice imo. Would it be possible to come up with a solution that is clear on this front, but still allows the developer to opt-in?

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Apr 1, 2017

Member

Great feedback. That's the first thing to figure out when working on this ticket :)

Member

mockitoguy commented Apr 1, 2017

Great feedback. That's the first thing to figure out when working on this ticket :)

@raphw

This comment has been minimized.

Show comment
Hide comment
@raphw

raphw Apr 1, 2017

Member

the way I saw mocking of static methods for myself was to only mock calls to static methods that happen within a mock. This would be useful, for example, for implementing spies in a situation like:

class Foo {
  String bar() { return Util.makeString(); }
}

With Mockito, I would like to see something like:

Foo foo = spy(new Foo());
doReturn("foo").when(foo).invokesStatic(Util.class).makeString();
assertThat(foo.bar()).isEqualTo("foo");

What do you think? As for a hack, I am missing a component in Byte Buddy which I was supposed to write for a customer at some point what did however not pull through. I am not currently in a position to spend so much time to build this component, due to my personal situation, but please prototype away. I think the API and spec work is crucial to make this a success.

Member

raphw commented Apr 1, 2017

the way I saw mocking of static methods for myself was to only mock calls to static methods that happen within a mock. This would be useful, for example, for implementing spies in a situation like:

class Foo {
  String bar() { return Util.makeString(); }
}

With Mockito, I would like to see something like:

Foo foo = spy(new Foo());
doReturn("foo").when(foo).invokesStatic(Util.class).makeString();
assertThat(foo.bar()).isEqualTo("foo");

What do you think? As for a hack, I am missing a component in Byte Buddy which I was supposed to write for a customer at some point what did however not pull through. I am not currently in a position to spend so much time to build this component, due to my personal situation, but please prototype away. I think the API and spec work is crucial to make this a success.

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Apr 3, 2017

Member

Great feedback! Thank you Rafael for sharing your thoughts.

Is the component you mentioned a lot of work in ByteBuddy? Is this something that we can ask the community to help out or you're the best person to take on?

I'd rather see more robust API but your API idea is certainly workable. Mocking statics should be rare and our API should be optimized for common use cases and not edge/rare cases. We can always iterate based on feedback :)

Here's my preference as of today:

//setup
mockStatic(Util.class);

//then
doAnswer().when(() -> Util.foo());
when(Util.foo()).thenReturn("foo");
verify(() -> Util.foo());
verify(never(), () -> Util.foo());

If we choose to support mocking static methods, I'd rather offer API that is robust enough to support all kinds of use cases. I like when tools are separated from policies and are opinionated but not dogmatic :) If we think that mocking static is disgusting, we can ship it in a different jar called "mockito-for-crappy-code", loosely following @TimvdLippe idea.

The main use cases for mocking statics I see are:

  • legacy code (I really, really want to write a unit test but I don't dare to change some ancient ugly code)
  • dealing with some awkward static 3rd party APIs. This scenario is currently workable by developing some injectable API layer on top of 3rd party statics. However, the workaround could be cumbersome and can spoil the clarity of codebase.

Without addressing above 2 use cases, developers around the world will be looking for help in tools like Powermockito, and such.

The biggest downside of offering static mocking is making it too easy to test crappy, procedural code full of static methods. We would remove a motivation to refactor the code into clean OO / DI.

Member

mockitoguy commented Apr 3, 2017

Great feedback! Thank you Rafael for sharing your thoughts.

Is the component you mentioned a lot of work in ByteBuddy? Is this something that we can ask the community to help out or you're the best person to take on?

I'd rather see more robust API but your API idea is certainly workable. Mocking statics should be rare and our API should be optimized for common use cases and not edge/rare cases. We can always iterate based on feedback :)

Here's my preference as of today:

//setup
mockStatic(Util.class);

//then
doAnswer().when(() -> Util.foo());
when(Util.foo()).thenReturn("foo");
verify(() -> Util.foo());
verify(never(), () -> Util.foo());

If we choose to support mocking static methods, I'd rather offer API that is robust enough to support all kinds of use cases. I like when tools are separated from policies and are opinionated but not dogmatic :) If we think that mocking static is disgusting, we can ship it in a different jar called "mockito-for-crappy-code", loosely following @TimvdLippe idea.

The main use cases for mocking statics I see are:

  • legacy code (I really, really want to write a unit test but I don't dare to change some ancient ugly code)
  • dealing with some awkward static 3rd party APIs. This scenario is currently workable by developing some injectable API layer on top of 3rd party statics. However, the workaround could be cumbersome and can spoil the clarity of codebase.

Without addressing above 2 use cases, developers around the world will be looking for help in tools like Powermockito, and such.

The biggest downside of offering static mocking is making it too easy to test crappy, procedural code full of static methods. We would remove a motivation to refactor the code into clean OO / DI.

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Apr 3, 2017

Contributor

Should we also send out a Tweet to gather some community insights?

Contributor

TimvdLippe commented Apr 3, 2017

Should we also send out a Tweet to gather some community insights?

@tmurakami

This comment has been minimized.

Show comment
Hide comment
@tmurakami

tmurakami Apr 3, 2017

Contributor

If this feature uses the Java Instrumentation API, it will not work on Android devices, so it might confuse Android developers.
If so, I think it should not be integrated into the mockito-core artifact.

Contributor

tmurakami commented Apr 3, 2017

If this feature uses the Java Instrumentation API, it will not work on Android devices, so it might confuse Android developers.
If so, I think it should not be integrated into the mockito-core artifact.

@bencampion

This comment has been minimized.

Show comment
Hide comment
@bencampion

bencampion Apr 3, 2017

I came across this issue by chance (I was just curious to know what you were planning for Mockito 3), but I'd like to share my experiences of working with static method mocking.

I used to work on several large codebases that made extensive use of static methods and used PowerMock (and PowerMockito) heavily for testing. My main memories of that time were that the tests were really slow to run (in some cases tests that use PowerMock were literally ten times slower than those that didn't) and that we ultimately had to remove all usages of PowerMock because it wasn't compatible with newer versions of Java. I also recall there being incompatibilities with some libraries were using that required special setup in the tests to make them pass, although I no longer remember the details of that.

I can understand concerns about promoting bad code, but as a Mockito user I'd be more worried about the impact on performance and robustness. Not all of the code we were using PowerMock for was legacy. Some of it was new code written by an inexperienced team (of which I was part of) and knowing that Mockito devs disapproved of our design patterns probably wouldn't have made any difference.

This was a few years ago now and techniques for mocking static methods may have improved significantly since then. If you think there's a performant and robust way to implement this feature then I'd welcome it (even though I'd hope I never have to use it).

bencampion commented Apr 3, 2017

I came across this issue by chance (I was just curious to know what you were planning for Mockito 3), but I'd like to share my experiences of working with static method mocking.

I used to work on several large codebases that made extensive use of static methods and used PowerMock (and PowerMockito) heavily for testing. My main memories of that time were that the tests were really slow to run (in some cases tests that use PowerMock were literally ten times slower than those that didn't) and that we ultimately had to remove all usages of PowerMock because it wasn't compatible with newer versions of Java. I also recall there being incompatibilities with some libraries were using that required special setup in the tests to make them pass, although I no longer remember the details of that.

I can understand concerns about promoting bad code, but as a Mockito user I'd be more worried about the impact on performance and robustness. Not all of the code we were using PowerMock for was legacy. Some of it was new code written by an inexperienced team (of which I was part of) and knowing that Mockito devs disapproved of our design patterns probably wouldn't have made any difference.

This was a few years ago now and techniques for mocking static methods may have improved significantly since then. If you think there's a performant and robust way to implement this feature then I'd welcome it (even though I'd hope I never have to use it).

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Apr 3, 2017

Member

Really good feedback, thank you guys. I helps us make good decisions about the features/API we plan.

Member

mockitoguy commented Apr 3, 2017

Really good feedback, thank you guys. I helps us make good decisions about the features/API we plan.

@szpak

This comment has been minimized.

Show comment
Hide comment
@szpak

szpak Apr 3, 2017

Member

In my opinion in general testing static methods is a bad idea and it would be better to refactor the code, however, sometimes it's something that is needed. For instance, testing a default method given() in mockito-java8 interface delegating to a static method in BDDMockito.given() the easiest solution I see is to generate in runtime a list of static methods in BDDMockito and execute parameterized test for every of them verifying that a corresponding method in WithBDDMockito interface delegates to it with proper parameters. In that case static method mocking would be beneficial. Maybe I will wait for PowerMockito for Mockito v2 to test Mockito itself...

Alternatively I would need to play with AOP and load-time weaving, code generation in runtime or some other not very pleasant to use technique (unless you have some good ideas how to do it easier).

Member

szpak commented Apr 3, 2017

In my opinion in general testing static methods is a bad idea and it would be better to refactor the code, however, sometimes it's something that is needed. For instance, testing a default method given() in mockito-java8 interface delegating to a static method in BDDMockito.given() the easiest solution I see is to generate in runtime a list of static methods in BDDMockito and execute parameterized test for every of them verifying that a corresponding method in WithBDDMockito interface delegates to it with proper parameters. In that case static method mocking would be beneficial. Maybe I will wait for PowerMockito for Mockito v2 to test Mockito itself...

Alternatively I would need to play with AOP and load-time weaving, code generation in runtime or some other not very pleasant to use technique (unless you have some good ideas how to do it easier).

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Apr 4, 2017

Member

@szpak, thank you for the feedback! Interesting use case.

I think we all agree that mocking statics is a bad practice and an anti-pattern in general. The question is whether we enforce the "no mocking statics" policy (e.g. not offer the feature in the tool) or let the user decide to enforce it or not (e.g. offer the feature).

Member

mockitoguy commented Apr 4, 2017

@szpak, thank you for the feedback! Interesting use case.

I think we all agree that mocking statics is a bad practice and an anti-pattern in general. The question is whether we enforce the "no mocking statics" policy (e.g. not offer the feature in the tool) or let the user decide to enforce it or not (e.g. offer the feature).

@Krazybug

This comment has been minimized.

Show comment
Hide comment
@Krazybug

Krazybug Apr 4, 2017

Please, consider the legacy code issue. Sometimes you don't have any choice.
Enforcing or not a practice shouldn't be the decision of a framework but of the team.
Also, for now, we're using Powermock as a workaround for this need but it's not compliant with the last versions of Mockito. We're blocked with 1.9.x version of Mockito.
It's another good reason to get this feature: Limit dependencies.

We already need all these libraries to get decent unit test in pure Java world:
JUnit,
JUnitParams as Junit Parametrized are awfull
AssertJ for expressive and clear assertions
Mockito ... and Powermock

and i shouldn't mention other peripheral libraries:
DBUnit,
SpringDBunit
JBehave, JGiven for BDD style ....

Then you can write all your testing stuf in Groovy and you only need ... Spock

Please! Let user choose. Othewise, why did you provide private method mocking in the recent versions athough it's considered as a "bad" practice for TDD purists ?

Finally, thanks for your great work. Mockito is really a nice framework

Krazybug commented Apr 4, 2017

Please, consider the legacy code issue. Sometimes you don't have any choice.
Enforcing or not a practice shouldn't be the decision of a framework but of the team.
Also, for now, we're using Powermock as a workaround for this need but it's not compliant with the last versions of Mockito. We're blocked with 1.9.x version of Mockito.
It's another good reason to get this feature: Limit dependencies.

We already need all these libraries to get decent unit test in pure Java world:
JUnit,
JUnitParams as Junit Parametrized are awfull
AssertJ for expressive and clear assertions
Mockito ... and Powermock

and i shouldn't mention other peripheral libraries:
DBUnit,
SpringDBunit
JBehave, JGiven for BDD style ....

Then you can write all your testing stuf in Groovy and you only need ... Spock

Please! Let user choose. Othewise, why did you provide private method mocking in the recent versions athough it's considered as a "bad" practice for TDD purists ?

Finally, thanks for your great work. Mockito is really a nice framework

@jlink

This comment has been minimized.

Show comment
Hide comment
@jlink

jlink Apr 4, 2017

Mocking static methods is a different use case, maybe that means it should be a different tool.

Answer the following questions (to yourself):

  • Do you want to support a double tool in the long term?
  • Does it use the same set of dependencies and only those?
  • Does mocking static methods fall nicely into the current design?

If you answer all 3 with "Yes", then go ahead. If at least one is a "No" consider the alternatives:

  • Build a Mockito extension
  • Build a new tool and call it Mackarita
  • Leave the other tool to someone else

Just my two cents.

jlink commented Apr 4, 2017

Mocking static methods is a different use case, maybe that means it should be a different tool.

Answer the following questions (to yourself):

  • Do you want to support a double tool in the long term?
  • Does it use the same set of dependencies and only those?
  • Does mocking static methods fall nicely into the current design?

If you answer all 3 with "Yes", then go ahead. If at least one is a "No" consider the alternatives:

  • Build a Mockito extension
  • Build a new tool and call it Mackarita
  • Leave the other tool to someone else

Just my two cents.

@karollewandowski

This comment has been minimized.

Show comment
Hide comment
@karollewandowski

karollewandowski Apr 4, 2017

I agree with most of you and think that mocking static methods is not a good idea.
If some method uses static method provided by legacy library/framework, then it can be easily wrapped in object and provided as mockable dependency. It's not perfect solution, but is educational - shows developers that writing too complex static util methods hurts and makes code hard to test. If Mockito had such feature, many of developers would consider it as acceptable practise and produce bad code.
Another issue is parallel tests execution. In my current project we used to use PowerMockito to mock static methods and after some time our tests started to fail because of concurrency issues:

  1. Thread A mocked static method X.y and stopped.
  2. Thread B mocked static method X.y and stopped.
  3. Thread A was awaken and run test - it failed because thread B overwritten static method's behaviour expected in test run by A.

We ended up with wrapping static methods in injectable objects and removed PowerMockito from project dependencies.

karollewandowski commented Apr 4, 2017

I agree with most of you and think that mocking static methods is not a good idea.
If some method uses static method provided by legacy library/framework, then it can be easily wrapped in object and provided as mockable dependency. It's not perfect solution, but is educational - shows developers that writing too complex static util methods hurts and makes code hard to test. If Mockito had such feature, many of developers would consider it as acceptable practise and produce bad code.
Another issue is parallel tests execution. In my current project we used to use PowerMockito to mock static methods and after some time our tests started to fail because of concurrency issues:

  1. Thread A mocked static method X.y and stopped.
  2. Thread B mocked static method X.y and stopped.
  3. Thread A was awaken and run test - it failed because thread B overwritten static method's behaviour expected in test run by A.

We ended up with wrapping static methods in injectable objects and removed PowerMockito from project dependencies.

@rdicroce

This comment has been minimized.

Show comment
Hide comment
@rdicroce

rdicroce Apr 4, 2017

I had asked about this at some point in the past and was told it was being considered. Glad to see that's actually happening.

In my opinion, support for mocking static methods is a good idea for the simple reason that the standard Java classes that ship with the JRE have tons of static methods. Consider the NIO.2 API as an example. If you want to read a file using that API, you might do something like:

Files.readAllLines(Paths.get("myfile"));

Now, is there a way to test this without mocking static methods? Obviously yes; you could make a IFiles interface and then make a JREFilesImpl that passes the calls through to the static methods. But that's a bunch of extra effort, it adds complexity to your code, and obscures the methods that are actually being called. In my opinion, I shouldn't need to do that to test my code's ability to properly read some file.

So +1 for the ability to mock static methods.

rdicroce commented Apr 4, 2017

I had asked about this at some point in the past and was told it was being considered. Glad to see that's actually happening.

In my opinion, support for mocking static methods is a good idea for the simple reason that the standard Java classes that ship with the JRE have tons of static methods. Consider the NIO.2 API as an example. If you want to read a file using that API, you might do something like:

Files.readAllLines(Paths.get("myfile"));

Now, is there a way to test this without mocking static methods? Obviously yes; you could make a IFiles interface and then make a JREFilesImpl that passes the calls through to the static methods. But that's a bunch of extra effort, it adds complexity to your code, and obscures the methods that are actually being called. In my opinion, I shouldn't need to do that to test my code's ability to properly read some file.

So +1 for the ability to mock static methods.

@karollewandowski

This comment has been minimized.

Show comment
Hide comment
@karollewandowski

karollewandowski Apr 4, 2017

Now, is there a way to test this without mocking static methods?

Well, in my opinion file name/path from your example is good candidate for passing it to method/setter/constructor instead of hardcoding it and you can just create test file in filesystem. Such Java API methods are considered as simple and reliable and there is no need to mock them like you wouldn't mock java.util.List if your method operated on data from given list. You would just create actual list with test data.

karollewandowski commented Apr 4, 2017

Now, is there a way to test this without mocking static methods?

Well, in my opinion file name/path from your example is good candidate for passing it to method/setter/constructor instead of hardcoding it and you can just create test file in filesystem. Such Java API methods are considered as simple and reliable and there is no need to mock them like you wouldn't mock java.util.List if your method operated on data from given list. You would just create actual list with test data.

@rdicroce

This comment has been minimized.

Show comment
Hide comment
@rdicroce

rdicroce Apr 4, 2017

There are at least 2 problems with that argument: a) you can't test behavior when exceptions occur, and b) you can't test behavior if the path is for a different OS. I realize the latter is an esoteric use case, but I actually am working on a project that is developed on Windows but runs exclusively on Linux.

rdicroce commented Apr 4, 2017

There are at least 2 problems with that argument: a) you can't test behavior when exceptions occur, and b) you can't test behavior if the path is for a different OS. I realize the latter is an esoteric use case, but I actually am working on a project that is developed on Windows but runs exclusively on Linux.

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Apr 4, 2017

Contributor

I think the recurring theme right now is: either you are pro or you are strictly against it. Personally, I am for clean tests and therefore consider static mocking very bad practice. However, our users apparently have usecases that require (intrusive?) solutions such as PowerMock. Given that these users opt for such solutions signals that the other solution would be no tests at all, and that is probably what we would never want.

Given that solutions like PowerMock are built not only on Mockito, but also other libraries, they can not be easily updated like Mockito can. This incentives Mockito to solve static mocking, if users opt-in for it.

Even though I inherently disagree with static mocking, I think it is only fair that we as Mockito developers offer users the option to opt-in a less intrusive and upgrade-compatible solution. Therefore, I would vouch for a different artifact (probably called mockito-legacy), which offers static mocking. This should signal our users that this is for legacy purposes and should be strictly discouraged, but at least gives them a way out. This particular solution would tick off all points of @jlink

In the end, no matter which outcome, there will always be disappointed developers. We will not be able to find a golden solution for this particular problem and I am fairly convinced it will never happen either.

We can always try it out with the new artifact and discontinue it later if it is significantly misused.

One sidenote: putting everything in a new artifact would scatter the mockito ecosystem. We should probably find a better solution for this, potentially in Mockito 3.

Contributor

TimvdLippe commented Apr 4, 2017

I think the recurring theme right now is: either you are pro or you are strictly against it. Personally, I am for clean tests and therefore consider static mocking very bad practice. However, our users apparently have usecases that require (intrusive?) solutions such as PowerMock. Given that these users opt for such solutions signals that the other solution would be no tests at all, and that is probably what we would never want.

Given that solutions like PowerMock are built not only on Mockito, but also other libraries, they can not be easily updated like Mockito can. This incentives Mockito to solve static mocking, if users opt-in for it.

Even though I inherently disagree with static mocking, I think it is only fair that we as Mockito developers offer users the option to opt-in a less intrusive and upgrade-compatible solution. Therefore, I would vouch for a different artifact (probably called mockito-legacy), which offers static mocking. This should signal our users that this is for legacy purposes and should be strictly discouraged, but at least gives them a way out. This particular solution would tick off all points of @jlink

In the end, no matter which outcome, there will always be disappointed developers. We will not be able to find a golden solution for this particular problem and I am fairly convinced it will never happen either.

We can always try it out with the new artifact and discontinue it later if it is significantly misused.

One sidenote: putting everything in a new artifact would scatter the mockito ecosystem. We should probably find a better solution for this, potentially in Mockito 3.

@marcingrzejszczak

This comment has been minimized.

Show comment
Hide comment
@marcingrzejszczak

marcingrzejszczak Apr 5, 2017

Collaborator

I disagree with @TimvdLippe. I don't believe that in an open source world you can "try it out with the new artifact and discontinue it later if it is significantly misused.". Once it's there in the library users will require it to be there. You'll have to support that feature etc. Either it's there or not. Not often can you easily deprecate sth and roll it back afterwards. Typically it requires a couple of release cycles.

@TimvdLippe you're mentioning this " However, our users apparently have usecases that require (intrusive?) solutions such as PowerMock". I have a comment on that. As an exercise from time to time I'm trying to answer PowerMock questions on StackOverflow. Typically someone says that there's a static method somewhere that they're calling and it does some logic that they want to stub. The question I often ask to those people is "Why are you doing this?". Would you really want to stub a call to StringUtils ? Another frequent problem is that the design is flawed and that's why one is looking for hacks. (e.g. http://stackoverflow.com/questions/37059406/how-can-i-get-powermock-to-return-the-expected-value-from-a-static-method/37066436#37066436 , http://stackoverflow.com/questions/37052069/program-termination-during-quartz-scheduler-verification-using-power-mockito/37066521#37066521 , http://stackoverflow.com/questions/31840964/powermock-private-method-with-null-pointer-exception/37066400#37066400 or http://stackoverflow.com/questions/32195928/getting-powermockito-to-mock-a-static-method-on-an-interface/32537392#32537392)

That's why I fully agree with @karollewandowski . "Every problem can be solved with a layer of abstraction" ;) You can't mock a static method? Ok then, wrap it in a class which you can stub. Is it wrong? Quite the contrary. Rarely should you call static methods directly and if you do, most likely these are utils that you don't want to stub. You should encapsulate the logic of a static method in an object that makes business sense to use it.

Summing it up giving people a tool to stub static methods is making them even easier to write bad code. Instead of thinking of how to fix the design they'll be able to make the design even worse.

Collaborator

marcingrzejszczak commented Apr 5, 2017

I disagree with @TimvdLippe. I don't believe that in an open source world you can "try it out with the new artifact and discontinue it later if it is significantly misused.". Once it's there in the library users will require it to be there. You'll have to support that feature etc. Either it's there or not. Not often can you easily deprecate sth and roll it back afterwards. Typically it requires a couple of release cycles.

@TimvdLippe you're mentioning this " However, our users apparently have usecases that require (intrusive?) solutions such as PowerMock". I have a comment on that. As an exercise from time to time I'm trying to answer PowerMock questions on StackOverflow. Typically someone says that there's a static method somewhere that they're calling and it does some logic that they want to stub. The question I often ask to those people is "Why are you doing this?". Would you really want to stub a call to StringUtils ? Another frequent problem is that the design is flawed and that's why one is looking for hacks. (e.g. http://stackoverflow.com/questions/37059406/how-can-i-get-powermock-to-return-the-expected-value-from-a-static-method/37066436#37066436 , http://stackoverflow.com/questions/37052069/program-termination-during-quartz-scheduler-verification-using-power-mockito/37066521#37066521 , http://stackoverflow.com/questions/31840964/powermock-private-method-with-null-pointer-exception/37066400#37066400 or http://stackoverflow.com/questions/32195928/getting-powermockito-to-mock-a-static-method-on-an-interface/32537392#32537392)

That's why I fully agree with @karollewandowski . "Every problem can be solved with a layer of abstraction" ;) You can't mock a static method? Ok then, wrap it in a class which you can stub. Is it wrong? Quite the contrary. Rarely should you call static methods directly and if you do, most likely these are utils that you don't want to stub. You should encapsulate the logic of a static method in an object that makes business sense to use it.

Summing it up giving people a tool to stub static methods is making them even easier to write bad code. Instead of thinking of how to fix the design they'll be able to make the design even worse.

@jlink

This comment has been minimized.

Show comment
Hide comment
@jlink

jlink Apr 5, 2017

@TimvdLippe Just one thing: If you create a new artifact like "mockito-legacy" communicate clearly if you're planning to support it mid/long-term or if it's just an experiment. Some people are willing to experiment together with you, others will be pissed off when you quit support after they have heavily used it.

jlink commented Apr 5, 2017

@TimvdLippe Just one thing: If you create a new artifact like "mockito-legacy" communicate clearly if you're planning to support it mid/long-term or if it's just an experiment. Some people are willing to experiment together with you, others will be pissed off when you quit support after they have heavily used it.

@marcingrzejszczak

This comment has been minimized.

Show comment
Hide comment
@marcingrzejszczak

marcingrzejszczak Apr 5, 2017

Collaborator

@rdicroce I completely disagree with this statement:

Now, is there a way to test this without mocking static methods? Obviously yes; you could make a IFiles interface and then make a JREFilesImpl that passes the calls through to the static methods. But that's a bunch of extra effort, it adds complexity to your code, and obscures the methods that are actually being called. In my opinion, I shouldn't need to do that to test my code's ability to properly read some file.

You're design is wrong. If you had

Files.readAllLines(Paths.get(myFilePath));

You could via a constructor inject myFilePath to point to your test resources. You don't even need to create any additional classes.

Collaborator

marcingrzejszczak commented Apr 5, 2017

@rdicroce I completely disagree with this statement:

Now, is there a way to test this without mocking static methods? Obviously yes; you could make a IFiles interface and then make a JREFilesImpl that passes the calls through to the static methods. But that's a bunch of extra effort, it adds complexity to your code, and obscures the methods that are actually being called. In my opinion, I shouldn't need to do that to test my code's ability to properly read some file.

You're design is wrong. If you had

Files.readAllLines(Paths.get(myFilePath));

You could via a constructor inject myFilePath to point to your test resources. You don't even need to create any additional classes.

@ChristianSchwarz

This comment has been minimized.

Show comment
Hide comment
@ChristianSchwarz

ChristianSchwarz Apr 5, 2017

Contributor

The Android community would like mock static methods, well at least me.
Why? : The Android SDK provides many static utility methods with classes like TextUtils. The downside is that there implementation is only provided on the device or emulator. When users run Unit-Test on there local machine they will get the famous "Method not mocked" exception.

The android.jar file that is used to run unit tests does not contain any actual code - that is provided by the Android system image on real devices. Instead, all methods throw exceptions (by default). This is to make sure your unit tests only test your code and do not depend on any particular behaviour of the Android platform (that you have not explicitly mocked e.g. using Mockito).

They are many workarounds for this issue like PowerMock or Roboelectric. Roboelectric is a great testing framework but it is damn slow and maintainance intensiv. PowerMock is not stable state to be used with Mockito 2. So I think at least Android users will love this feature.

Contributor

ChristianSchwarz commented Apr 5, 2017

The Android community would like mock static methods, well at least me.
Why? : The Android SDK provides many static utility methods with classes like TextUtils. The downside is that there implementation is only provided on the device or emulator. When users run Unit-Test on there local machine they will get the famous "Method not mocked" exception.

The android.jar file that is used to run unit tests does not contain any actual code - that is provided by the Android system image on real devices. Instead, all methods throw exceptions (by default). This is to make sure your unit tests only test your code and do not depend on any particular behaviour of the Android platform (that you have not explicitly mocked e.g. using Mockito).

They are many workarounds for this issue like PowerMock or Roboelectric. Roboelectric is a great testing framework but it is damn slow and maintainance intensiv. PowerMock is not stable state to be used with Mockito 2. So I think at least Android users will love this feature.

@dbacinski

This comment has been minimized.

Show comment
Hide comment
@dbacinski

dbacinski Apr 5, 2017

@ChristianSchwarz I do not agree, using Android static utils or Android framework in your business logic is a bad smell. Well structured Android app do not need mocking static methods, nor Roboelectric. Calls to Android framework must be abstracted and current Mockito implementation helps to enforce that. If you want to test view layer then go with Instrumentation tests and Espresso.

dbacinski commented Apr 5, 2017

@ChristianSchwarz I do not agree, using Android static utils or Android framework in your business logic is a bad smell. Well structured Android app do not need mocking static methods, nor Roboelectric. Calls to Android framework must be abstracted and current Mockito implementation helps to enforce that. If you want to test view layer then go with Instrumentation tests and Espresso.

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Apr 5, 2017

Member

@dbacinski, I am not an expert on Android so bear with me, please :) Adding extra layer introduces more method calls which can be problematic on large apps that hit dex method limit, right?

Member

mockitoguy commented Apr 5, 2017

@dbacinski, I am not an expert on Android so bear with me, please :) Adding extra layer introduces more method calls which can be problematic on large apps that hit dex method limit, right?

@dbacinski

This comment has been minimized.

Show comment
Hide comment
@dbacinski

dbacinski Apr 5, 2017

@szczepiq you are right that you need additional methods to abstract Android apis, but this is a cost of good architecture. Dex method limit is not a huge problem anymore because now there is native platform support for multidex apps.

dbacinski commented Apr 5, 2017

@szczepiq you are right that you need additional methods to abstract Android apis, but this is a cost of good architecture. Dex method limit is not a huge problem anymore because now there is native platform support for multidex apps.

@rdicroce

This comment has been minimized.

Show comment
Hide comment
@rdicroce

rdicroce Apr 5, 2017

@marcingrzejszczak Your response a) does not address the call to readAllLines(), and b) does not address either of the issues I raised in #1013 (comment)

To all: it seems to me there's a fundamental divide in this thread between people who are of the opinion that it's fine to just make a wrapper class for static calls, and people who are of the opinion that wrapper classes add bloat and shouldn't be necessary. I don't see either side convincing the other, so I'm not sure where that leaves us.

rdicroce commented Apr 5, 2017

@marcingrzejszczak Your response a) does not address the call to readAllLines(), and b) does not address either of the issues I raised in #1013 (comment)

To all: it seems to me there's a fundamental divide in this thread between people who are of the opinion that it's fine to just make a wrapper class for static calls, and people who are of the opinion that wrapper classes add bloat and shouldn't be necessary. I don't see either side convincing the other, so I'm not sure where that leaves us.

@marcingrzejszczak

This comment has been minimized.

Show comment
Hide comment
@marcingrzejszczak

marcingrzejszczak Apr 5, 2017

Collaborator

Your response a) does not address the call to readAllLines(), and b) does not address either of the issues I raised in #1013 (comment)

@rdicroce I haven't explicitly but the answer is simple

a) does not address the call to readAllLines(),

If the path is a parameter you don't have to mock the call at all. You can pass

a) a path that exists - to test the positive scenario
b) a path that doesn't exist - that way it will blow up and you'll test an exception

. I realize the latter is an esoteric use case, but I actually am working on a project that is developed on Windows but runs exclusively on Linux.

You can write a couple of tests that are exclusively executed depending on the OS (example for Windows - http://stackoverflow.com/questions/23410738/run-unit-tests-only-on-windows ). Again, if you use parameters instead of magic values then you can do basically whatever you want.

To all: it seems to me there's a fundamental divide in this thread between people who are of the opinion that it's fine to just make a wrapper class for static calls, and people who are of the opinion that wrapper classes add bloat and shouldn't be necessary. I don't see either side convincing the other, so I'm not sure where that leaves us.

I'd say that the divide is between people who want to design their code properly and those who want to take the path of least resistance (which isn't a wrong choice sometimes).

Collaborator

marcingrzejszczak commented Apr 5, 2017

Your response a) does not address the call to readAllLines(), and b) does not address either of the issues I raised in #1013 (comment)

@rdicroce I haven't explicitly but the answer is simple

a) does not address the call to readAllLines(),

If the path is a parameter you don't have to mock the call at all. You can pass

a) a path that exists - to test the positive scenario
b) a path that doesn't exist - that way it will blow up and you'll test an exception

. I realize the latter is an esoteric use case, but I actually am working on a project that is developed on Windows but runs exclusively on Linux.

You can write a couple of tests that are exclusively executed depending on the OS (example for Windows - http://stackoverflow.com/questions/23410738/run-unit-tests-only-on-windows ). Again, if you use parameters instead of magic values then you can do basically whatever you want.

To all: it seems to me there's a fundamental divide in this thread between people who are of the opinion that it's fine to just make a wrapper class for static calls, and people who are of the opinion that wrapper classes add bloat and shouldn't be necessary. I don't see either side convincing the other, so I'm not sure where that leaves us.

I'd say that the divide is between people who want to design their code properly and those who want to take the path of least resistance (which isn't a wrong choice sometimes).

@pioorg

This comment has been minimized.

Show comment
Hide comment
@pioorg

pioorg Apr 5, 2017

I'd say that the divide is between people who want to design their code properly and those who want to take the path of least resistance.

Static members aren't something good, I hope it's quite obvious. But Java has them and will support them, whether we like it or not. So I'd say that educating and influencing is good, forcing might be not. It's like "oooh, your code is using statics, so it can't be tested with Mockito".
(Not to mention that some people would like to have nice unclebobish tests but aren't allowed to refactor the code they have to maintain. I know it's sick, but hey, this is reality sometimes.)

pioorg commented Apr 5, 2017

I'd say that the divide is between people who want to design their code properly and those who want to take the path of least resistance.

Static members aren't something good, I hope it's quite obvious. But Java has them and will support them, whether we like it or not. So I'd say that educating and influencing is good, forcing might be not. It's like "oooh, your code is using statics, so it can't be tested with Mockito".
(Not to mention that some people would like to have nice unclebobish tests but aren't allowed to refactor the code they have to maintain. I know it's sick, but hey, this is reality sometimes.)

@pioorg

This comment has been minimized.

Show comment
Hide comment
@pioorg

pioorg Apr 11, 2017

@kkapelon: I also wish all Java code was pure OO design and that everyone actually knows what that means ;-) I hope such a huge warning "you're doing it wrong" could educate and maybe convert a few folks.

Maybe I am missing something, but I really would like some examples on how legacy code cannot be refactored to remove the static dependency.

It's not about the code itself. I've seen big name companies enforcing policies in which before bugfixing, the case has to be replicated in tests. (Kinda TDD, they claim sometimes.) Then, once the test fails just like the ticket describes, the support team is allowed to change the code to make it pass (and sadly, usually minimum changes). It's rather difficult to make a test to cover statics if your testing tool set doesn't allow that. Chicken and egg problem.

It's like all these The Daily WTF stories in which "don't change it, it ain't broke, 'couse it works!", so the code is like Java 1.4...

pioorg commented Apr 11, 2017

@kkapelon: I also wish all Java code was pure OO design and that everyone actually knows what that means ;-) I hope such a huge warning "you're doing it wrong" could educate and maybe convert a few folks.

Maybe I am missing something, but I really would like some examples on how legacy code cannot be refactored to remove the static dependency.

It's not about the code itself. I've seen big name companies enforcing policies in which before bugfixing, the case has to be replicated in tests. (Kinda TDD, they claim sometimes.) Then, once the test fails just like the ticket describes, the support team is allowed to change the code to make it pass (and sadly, usually minimum changes). It's rather difficult to make a test to cover statics if your testing tool set doesn't allow that. Chicken and egg problem.

It's like all these The Daily WTF stories in which "don't change it, it ain't broke, 'couse it works!", so the code is like Java 1.4...

@lukaseder

This comment has been minimized.

Show comment
Hide comment
@lukaseder

lukaseder Apr 11, 2017

This would be great! We could finally stop bikeshedding "static is eeevil" because not testable.

lukaseder commented Apr 11, 2017

This would be great! We could finally stop bikeshedding "static is eeevil" because not testable.

@pioorg

This comment has been minimized.

Show comment
Hide comment
@pioorg

pioorg Apr 11, 2017

We could finally stop bikeshedding "static is eeevil" because not testable.

Well, IMHO static is eeevil in many cases not only because not testable :-D

pioorg commented Apr 11, 2017

We could finally stop bikeshedding "static is eeevil" because not testable.

Well, IMHO static is eeevil in many cases not only because not testable :-D

@helmbold

This comment has been minimized.

Show comment
Hide comment
@helmbold

helmbold Apr 11, 2017

The mocking framework shouldn't restrict the design options - it's just not the job of a mocking framework! I've seen a lot of code polluted with mocking-only interfaces. And there are a lot of cases where a simple static method is a perfect design decision. final is also rarely used, because mocking frameworks (besides the great JMockit) have trouble with it.

helmbold commented Apr 11, 2017

The mocking framework shouldn't restrict the design options - it's just not the job of a mocking framework! I've seen a lot of code polluted with mocking-only interfaces. And there are a lot of cases where a simple static method is a perfect design decision. final is also rarely used, because mocking frameworks (besides the great JMockit) have trouble with it.

@trumpetx

This comment has been minimized.

Show comment
Hide comment
@trumpetx

trumpetx Apr 11, 2017

DONT DO IT! PowerMockito exists for those cases where you absolutely need to mock static methods. Right now Mockito conforms to the commonly held opinion that static is for methods that are testable without mocks (think: StringUtils). Static for other stuff should be discouraged (there are exceptions, obviously) - in those situations, encapsulating the static calls in a protected @VisibleForTesting method should suffice for unit testing and for those other "legacy or not" situations, there's always PowerMockito.

trumpetx commented Apr 11, 2017

DONT DO IT! PowerMockito exists for those cases where you absolutely need to mock static methods. Right now Mockito conforms to the commonly held opinion that static is for methods that are testable without mocks (think: StringUtils). Static for other stuff should be discouraged (there are exceptions, obviously) - in those situations, encapsulating the static calls in a protected @VisibleForTesting method should suffice for unit testing and for those other "legacy or not" situations, there's always PowerMockito.

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Apr 11, 2017

Contributor

@trumpetx I just wanted to note that the reason we are investigating this change is that some companies are in the legacy code situation, where adopting PowerMockito would be too disruptive. Therefore, these companies would be unable to test their classes. Even if they could adopt PowerMockito, the tests become significantly more difficult to understand, just to be able to test their legacy code.

The question is: should we as Mockito give the opt-in for these situations, as other options are too invasive? I think every Mockito core developer agrees that static mocking is bad and should not be done, but sadly we can not dictate what the industry does or has to deal with.

Just wanted to give perspective on why we are opening this issue.

Contributor

TimvdLippe commented Apr 11, 2017

@trumpetx I just wanted to note that the reason we are investigating this change is that some companies are in the legacy code situation, where adopting PowerMockito would be too disruptive. Therefore, these companies would be unable to test their classes. Even if they could adopt PowerMockito, the tests become significantly more difficult to understand, just to be able to test their legacy code.

The question is: should we as Mockito give the opt-in for these situations, as other options are too invasive? I think every Mockito core developer agrees that static mocking is bad and should not be done, but sadly we can not dictate what the industry does or has to deal with.

Just wanted to give perspective on why we are opening this issue.

@trumpetx

This comment has been minimized.

Show comment
Hide comment
@trumpetx

trumpetx Apr 11, 2017

@TimvdLippe , I can't see how Powermockito would be disruptive for tests (certainly no more disruptive than adding MockitoStaticWhatever.)

Additionally, too difficult to understand? Seriously? If someone has static mocks in their tests, and they're too difficult to understand, they probably need to find a new career. I'm not saying this to be mean spirited or whatnot, but it sounds like you're just making reasons to adopt this change.

trumpetx commented Apr 11, 2017

@TimvdLippe , I can't see how Powermockito would be disruptive for tests (certainly no more disruptive than adding MockitoStaticWhatever.)

Additionally, too difficult to understand? Seriously? If someone has static mocks in their tests, and they're too difficult to understand, they probably need to find a new career. I'm not saying this to be mean spirited or whatnot, but it sounds like you're just making reasons to adopt this change.

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Apr 11, 2017

Contributor

Also for reference, I cross-posted this issue on Reddit and there some other arguments/viewpoints posted there: https://www.reddit.com/r/java/comments/64pdk1/mockito_discussion_on_introduction_of_mocking/

Contributor

TimvdLippe commented Apr 11, 2017

Also for reference, I cross-posted this issue on Reddit and there some other arguments/viewpoints posted there: https://www.reddit.com/r/java/comments/64pdk1/mockito_discussion_on_introduction_of_mocking/

@pioorg

This comment has been minimized.

Show comment
Hide comment
@pioorg

pioorg Apr 11, 2017

@TimvdLippe
Thanks for your link. The recent posts have led me to another...
http://mvnrepository.com/artifact/org.powermock/powermock-api-mockito/usages
I'm just wondering why so many of them...

pioorg commented Apr 11, 2017

@TimvdLippe
Thanks for your link. The recent posts have led me to another...
http://mvnrepository.com/artifact/org.powermock/powermock-api-mockito/usages
I'm just wondering why so many of them...

@AnEmortalKid

This comment has been minimized.

Show comment
Hide comment
@AnEmortalKid

AnEmortalKid Apr 12, 2017

I haven't had experience with ByteBuddy but I'd be willing to give it a shot. As far as names, how do we feel about mockito-gore to really indicate you need a strong reason to use this jar?

AnEmortalKid commented Apr 12, 2017

I haven't had experience with ByteBuddy but I'd be willing to give it a shot. As far as names, how do we feel about mockito-gore to really indicate you need a strong reason to use this jar?

@pioorg

This comment has been minimized.

Show comment
Hide comment
@pioorg

pioorg Apr 19, 2017

Just a few more words from me, maybe last ones ;-)
There's a method in Mockito:
org.mockito.Matchers#isNull(java.lang.Class<T>)
Isn't NULL considered to be "The Billion Dollar Mistake"? Shouldn't the programmers all around the world be educated that passing a null reference (instead of a java.util.Optional) is a very bad practice?
Maybe this isNull method should be deprecated and eventually removed in future versions?

I've seen a lot of code being polluted by if (reference == null) and catch (NullPointerException e)... More than by "incorrect use of statics". (But YMMV.)
If supporting statics is going to create such bad habits among programmers AND Mockito is meant to be a forced education, then I guess support for NULLs should be removed in future versions and people should write wrappers to handle NULLs from their legacy code, as NULL is worse habit IMHO. ;-)

Unless Mockito is meant to be a testing/mocking framework, not forced education course.
Thank you.

pioorg commented Apr 19, 2017

Just a few more words from me, maybe last ones ;-)
There's a method in Mockito:
org.mockito.Matchers#isNull(java.lang.Class<T>)
Isn't NULL considered to be "The Billion Dollar Mistake"? Shouldn't the programmers all around the world be educated that passing a null reference (instead of a java.util.Optional) is a very bad practice?
Maybe this isNull method should be deprecated and eventually removed in future versions?

I've seen a lot of code being polluted by if (reference == null) and catch (NullPointerException e)... More than by "incorrect use of statics". (But YMMV.)
If supporting statics is going to create such bad habits among programmers AND Mockito is meant to be a forced education, then I guess support for NULLs should be removed in future versions and people should write wrappers to handle NULLs from their legacy code, as NULL is worse habit IMHO. ;-)

Unless Mockito is meant to be a testing/mocking framework, not forced education course.
Thank you.

@filiphr

This comment has been minimized.

Show comment
Hide comment
@filiphr

filiphr Apr 20, 2017

I would like to add some comments from my own experience. I completely understand the people that are saying to add an abstraction to the static methods that you are invoking. However, in a legacy code that is expensive and requires a lot of effort.

Additionally, there are frameworks like GWT that support polymorphism, but not like you would think. One needs to call a static method that will do the creation of the object on runtime and runtime there is Javascript. There ware ways that you can write tests that will emulated the client code for GWT, but they are really slow. Currently, we use PowerMockito with Mockito 1.9.5, because we can't update to a later Mockito. However, if Mockito had this support it would be much easier to do the testing.

filiphr commented Apr 20, 2017

I would like to add some comments from my own experience. I completely understand the people that are saying to add an abstraction to the static methods that you are invoking. However, in a legacy code that is expensive and requires a lot of effort.

Additionally, there are frameworks like GWT that support polymorphism, but not like you would think. One needs to call a static method that will do the creation of the object on runtime and runtime there is Javascript. There ware ways that you can write tests that will emulated the client code for GWT, but they are really slow. Currently, we use PowerMockito with Mockito 1.9.5, because we can't update to a later Mockito. However, if Mockito had this support it would be much easier to do the testing.

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Apr 29, 2017

Contributor

This discussion has been very active and helpful, thank you everyone for participating!

I think we have sufficient background information to make a final decision @mockito/core Shall we discuss this on Slack and write a wiki page on our full analysis and final verdict? Then we can inform the community for a final review (of a potential implementation) and round up this feature.

Contributor

TimvdLippe commented Apr 29, 2017

This discussion has been very active and helpful, thank you everyone for participating!

I think we have sufficient background information to make a final decision @mockito/core Shall we discuss this on Slack and write a wiki page on our full analysis and final verdict? Then we can inform the community for a final review (of a potential implementation) and round up this feature.

@mattnelson

This comment has been minimized.

Show comment
Hide comment
@mattnelson

mattnelson Jul 21, 2017

Shall we discuss this on Slack and write a wiki page on our full analysis and final verdict? Then we can inform the community for a final review (of a potential implementation) and round up this feature.

Has the verdict on static support been reached?

mattnelson commented Jul 21, 2017

Shall we discuss this on Slack and write a wiki page on our full analysis and final verdict? Then we can inform the community for a final review (of a potential implementation) and round up this feature.

Has the verdict on static support been reached?

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Jul 22, 2017

Contributor

We have not. Every single core member has been swamped with various IRL activities. Probably this fall we will be back in full force. This is one of the features that we will be addressing 😄

Contributor

TimvdLippe commented Jul 22, 2017

We have not. Every single core member has been swamped with various IRL activities. Probably this fall we will be back in full force. This is one of the features that we will be addressing 😄

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Jul 23, 2017

Member

The team was not actively working on this. The ticket is marked "please contribute" for a reason ;) Currently I work with @thekingnothing on creating Mockito APIs so that Powermock can cleanly integrate (#1110) this will effectively provide reasonable support for static mocking.

Hope this clarifies the status! Thank you for reminder - this is useful for us to manage our priorities.

Member

mockitoguy commented Jul 23, 2017

The team was not actively working on this. The ticket is marked "please contribute" for a reason ;) Currently I work with @thekingnothing on creating Mockito APIs so that Powermock can cleanly integrate (#1110) this will effectively provide reasonable support for static mocking.

Hope this clarifies the status! Thank you for reminder - this is useful for us to manage our priorities.

@ogregoire

This comment has been minimized.

Show comment
Hide comment
@ogregoire

ogregoire Jul 27, 2017

Static methods are totally OK. You just have to know when to use them.

I think the following code is entirely correct and sufficiently abstracted for the purpose of a modern-day application:

// guava imports
class ApplicationResources {
  // points to a file in the .war
  private static final CharSource DEFAULT_CONFIG_SOURCE = Resources.asCharSource(Resources.get("etc/default.config"), UTF_8);
  static CharSource defaultConfigSource() {
    return DEFAULT_CONFIG_SOURCE;
  }
}

I need to access a static value in my application. But for the purpose of proper testing, I need to test various default values, so it makes sense to provide another CharSource during the tests than at runtime.

Just imagine that this is embedded in some JEE application. There is no sense in writing this method as non-static. I could write new ApplicationResources().defaultConfigSource(). What is that code? Is it real? Others might imagine I make this class a (singleton) bean. A bean requires the class to be public. I don't want any of its method to be accessed outside of the package, so there are no reasons to make this class public, so this class has no reasons to be a bean.

However I still have to test classes that use this class. But I'm stuck in the testing because I just can't mock its methods (besides using all the workarounds mentioned above).

I agree that mocking static method should be used very, very rarely, but it should exist for those rare cases.

ogregoire commented Jul 27, 2017

Static methods are totally OK. You just have to know when to use them.

I think the following code is entirely correct and sufficiently abstracted for the purpose of a modern-day application:

// guava imports
class ApplicationResources {
  // points to a file in the .war
  private static final CharSource DEFAULT_CONFIG_SOURCE = Resources.asCharSource(Resources.get("etc/default.config"), UTF_8);
  static CharSource defaultConfigSource() {
    return DEFAULT_CONFIG_SOURCE;
  }
}

I need to access a static value in my application. But for the purpose of proper testing, I need to test various default values, so it makes sense to provide another CharSource during the tests than at runtime.

Just imagine that this is embedded in some JEE application. There is no sense in writing this method as non-static. I could write new ApplicationResources().defaultConfigSource(). What is that code? Is it real? Others might imagine I make this class a (singleton) bean. A bean requires the class to be public. I don't want any of its method to be accessed outside of the package, so there are no reasons to make this class public, so this class has no reasons to be a bean.

However I still have to test classes that use this class. But I'm stuck in the testing because I just can't mock its methods (besides using all the workarounds mentioned above).

I agree that mocking static method should be used very, very rarely, but it should exist for those rare cases.

@raphw

This comment has been minimized.

Show comment
Hide comment
@raphw

raphw Jul 30, 2017

Member

Personally, I plan to add this feature. I still need to do some ground work in Byte Buddy to make this feasible. It is not as easy as I hoped, unfortunately and it might take some time. The same goes for the seamless integration with PowerMock.

If anybody would be willing to sponsor the development, I am happy to use paid hours, within my open source commitment, this might take a few more month to complete but I am certain to finish this at some point. Java 9 compatibility takes priority at the moment.

Member

raphw commented Jul 30, 2017

Personally, I plan to add this feature. I still need to do some ground work in Byte Buddy to make this feasible. It is not as easy as I hoped, unfortunately and it might take some time. The same goes for the seamless integration with PowerMock.

If anybody would be willing to sponsor the development, I am happy to use paid hours, within my open source commitment, this might take a few more month to complete but I am certain to finish this at some point. Java 9 compatibility takes priority at the moment.

@moltmann

This comment has been minimized.

Show comment
Hide comment
@moltmann

moltmann Dec 12, 2017

Contributor

tl;dr: mocking static methods can be implemented mostly like regular mocking with a few quirks when spying on static methods with side-effects.

I implemented a prototype of this for the dexmaker based mocking used on Android. Examples and tests can be found in MockStatic.java in the diff.

I summarized my observations on a wiki page.

Contributor

moltmann commented Dec 12, 2017

tl;dr: mocking static methods can be implemented mostly like regular mocking with a few quirks when spying on static methods with side-effects.

I implemented a prototype of this for the dexmaker based mocking used on Android. Examples and tests can be found in MockStatic.java in the diff.

I summarized my observations on a wiki page.

@moltmann

This comment has been minimized.

Show comment
Hide comment
@moltmann

moltmann Dec 14, 2017

Contributor

I think I found a nicer version for mocking static methods via doReturn().when(). So the total interface for static methods could be designed as:

try (StaticMockingInProgress<MyClass> token = spyStatic(MyClass.class)) {
    when(MyClass.aStaticMethod()).thenReturn(“mockedValue”);
    assertEquals(“mockedValue”, MyClass.aStaticMethod());
    
    doReturn(“fakeValue”).whenStatic(MyClass.class).anotherStaticMethod();
    assertEquals(“fakeValue”, MyClass.anotherStaticMethod());
}

assertEquals(“originalValue”, MyClass.aStaticMethod());
assertEquals(“originalValue”, MyClass.anotherStaticMethod());

In Summary we could limit the necessary changed to:

  • mockStatic/spyStatic instead of mock/spy. This is required as mockStatic/spyStatic needs to return a token to define the scope of the mocking instead of a mocked object
  • doReturn(...).whenStatic(Class) instead of doReturn(...).when(Object). This is necessary as when mocking a static method, there is no mocked object
Contributor

moltmann commented Dec 14, 2017

I think I found a nicer version for mocking static methods via doReturn().when(). So the total interface for static methods could be designed as:

try (StaticMockingInProgress<MyClass> token = spyStatic(MyClass.class)) {
    when(MyClass.aStaticMethod()).thenReturn(“mockedValue”);
    assertEquals(“mockedValue”, MyClass.aStaticMethod());
    
    doReturn(“fakeValue”).whenStatic(MyClass.class).anotherStaticMethod();
    assertEquals(“fakeValue”, MyClass.anotherStaticMethod());
}

assertEquals(“originalValue”, MyClass.aStaticMethod());
assertEquals(“originalValue”, MyClass.anotherStaticMethod());

In Summary we could limit the necessary changed to:

  • mockStatic/spyStatic instead of mock/spy. This is required as mockStatic/spyStatic needs to return a token to define the scope of the mocking instead of a mocked object
  • doReturn(...).whenStatic(Class) instead of doReturn(...).when(Object). This is necessary as when mocking a static method, there is no mocked object
@moltmann

This comment has been minimized.

Show comment
Hide comment
@moltmann

moltmann Dec 20, 2017

Contributor

Addendum to previous comment. I think we should also have a resetStatic(Class<T> class) method. Then using the StaticMockingInProgress object becomes optional but recommended.

Contributor

moltmann commented Dec 20, 2017

Addendum to previous comment. I think we should also have a resetStatic(Class<T> class) method. Then using the StaticMockingInProgress object becomes optional but recommended.

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Jan 28, 2018

Member

Hey @moltmann, thank you for taking a stab at the prototype!

Given that we provide an API for mocking statics in mockito-core, will you be able to come up with the implementation in dexmaker?

Can you implement static mocking in dexmaker right now, with current version of mockito-core API? For example, can you add MockitoStatic class to dexmaker and offer static mocking to dexmaker users? If not, what are the problems with mockito-core API that block you.

Member

mockitoguy commented Jan 28, 2018

Hey @moltmann, thank you for taking a stab at the prototype!

Given that we provide an API for mocking statics in mockito-core, will you be able to come up with the implementation in dexmaker?

Can you implement static mocking in dexmaker right now, with current version of mockito-core API? For example, can you add MockitoStatic class to dexmaker and offer static mocking to dexmaker users? If not, what are the problems with mockito-core API that block you.

@moltmann

This comment has been minimized.

Show comment
Hide comment
@moltmann

moltmann Mar 21, 2018

Contributor

I am trying to sum up all discussion I had and what I learnt from this thread.

I updated my prototype at https://github.com/moltmann/dexmaker/tree/staticMock using

  • whenStatic
  • explicit finishStaticMocking

Stubbing

How does "when" work for static methods?

when(λambda)

Suggested APIs

when(() -> MyClass.myMethod()).thenReturn("mocked return value");

doReturn("mocked return value").when(() -> MyClass.myMethod());

verify(() -> MyClass.myMethod());

Problems

Not clear what method is called

If the lambda calls two methods, e.g.:

doReturn("mocked return value").when(() -> MyClass.firstMethod().secondMethod());

or

doReturn("mocked return value").when(() -> {
    Object o = MyClass.firstMethod();
    return o.secondMethod()
});

Which method should we set up mocking for? MyClass#firstMethod() or ?#secondMethod().

  • We need to make sure the return type matches the parameter of doReturn. We only know the return type of the the lambda, i.e. the return value of secondMethod. Hence we have to set up mocking for the second method.
    We don't set up mocking for MyClass.firstMethod(). What should MyClass.firstMethod() do? Call the real method? Return null? To make the code above work MyClass.firstMethod() has to call the real method. But this might have side effects. The point of the doReturn(...).when(...) syntax is to avoid side-effects.
  • The second method is most likely an instance (non static) method. Should we allow setup for instance methods using the when-with-lambda?

I think we need to enforce only a single mocked method call on a mock in the lambda. Hence we have to listen to all possible methods calls, and intercept them. Once a second one or one of the wrong return type is intercepted we need to throw an exception. Hence above examples will throw an exception.

Possible extension

One possible extension would be do declare a restriction on what to intercept like

doReturn("mocked return value").when(MyOtherClass.class, () -> {
    MyOtherClass o = MyClass.firstMethod();
    return o.secondMethod()
});

Which declares that we want to set up mocking for MyOtherClass, not MyClass.

Conflict between old and new doReturn-when

doReturn(() -> "A").when(() -> MyClass.firstMethod());

In above case Mockito needs to decide if this is the new setup method or an incomplete version of

Supplier<String> m = mock(Supplier.class);

doReturn(() -> "A").when(m).get();

it can do this by checking if the parameter to m is a mock during runtime, but the when(λambda) API needs to support both cases, hence when(λambda) has to return a lambda type, i.e.

LambdaStubber<Supplier<T>> doReturn(Supplier<T> ret) {
    return new LambdaStubber<Supplier<T>>(ret);
}

public static class LambdaStubber<T> implements Stubber {
    Supplier<T> when(Supplier<T>) {
        ....
    }
}
Possible extension

This issue might be mitigated by introducing

doReturn(() -> "A").whenStatic(() -> MyClass.firstMethod());

T whenStatic(Class<T>)

doReturn("mocked return value").whenStatic(MyClass.class).myStaticMethod();

verifyStatic(MyClass.class).myStaticMethod();

doReturn(...).whenStatic(MyClass.class) returns a mock instance of MyClass with all instance methods throwing a stubbing exception. So the user will not accidentally call a instance method on this special stubbing-mock.

Problems

doReturn(...).whenStatic(MyClass.class) returns a mock instance of MyClass. Hence calling doReturn(...).whenStatic(MyClass.class).myStaticMethod() is equivalent to

MyClass m = mock(Myclass.class);
m.myStaticMethod();

While it is valid Java to call static methods on instances, all IDE code checkers, linters and tools like errorprone will light up like a christmas tree and tell you not to do it.

Life-time of static mocking

Regular instance mocking is tied to an object's lifetime. Static methods do not have such a life-time, hence when to know when to reset the methods and stop collecting invocations.

mockStatic(Class) -> finishStaticMocking(Class)

This is the most basic solution. The advantage is that this works with every possible test-framework and if someone wants to have a nicer API they can e.g. build a AutoClosable that automatically calls finishStaticMocking(...) when a try-with-resources block is exited.

MockitoSessions

As far as I understood there can only be one session at a time. Hence scenarios with interleaved life-cycles of static mocking can not be supported. Example:

mockStatic(A.class)
mockStatic(B.class)

when(A.m()).thenReturn("mockA")
when(B.m()).thenReturn("mockB")

assertEquals("mockA", A.m());

finishStaticMocking(A.class)

assertEquals("mockB", B.m());

finishStaticMocking(A.class)

Of course if the user wants to use sessions (or other session like tools, such as the Mockito JUnit runner) they should apply to static mocking too.

Contributor

moltmann commented Mar 21, 2018

I am trying to sum up all discussion I had and what I learnt from this thread.

I updated my prototype at https://github.com/moltmann/dexmaker/tree/staticMock using

  • whenStatic
  • explicit finishStaticMocking

Stubbing

How does "when" work for static methods?

when(λambda)

Suggested APIs

when(() -> MyClass.myMethod()).thenReturn("mocked return value");

doReturn("mocked return value").when(() -> MyClass.myMethod());

verify(() -> MyClass.myMethod());

Problems

Not clear what method is called

If the lambda calls two methods, e.g.:

doReturn("mocked return value").when(() -> MyClass.firstMethod().secondMethod());

or

doReturn("mocked return value").when(() -> {
    Object o = MyClass.firstMethod();
    return o.secondMethod()
});

Which method should we set up mocking for? MyClass#firstMethod() or ?#secondMethod().

  • We need to make sure the return type matches the parameter of doReturn. We only know the return type of the the lambda, i.e. the return value of secondMethod. Hence we have to set up mocking for the second method.
    We don't set up mocking for MyClass.firstMethod(). What should MyClass.firstMethod() do? Call the real method? Return null? To make the code above work MyClass.firstMethod() has to call the real method. But this might have side effects. The point of the doReturn(...).when(...) syntax is to avoid side-effects.
  • The second method is most likely an instance (non static) method. Should we allow setup for instance methods using the when-with-lambda?

I think we need to enforce only a single mocked method call on a mock in the lambda. Hence we have to listen to all possible methods calls, and intercept them. Once a second one or one of the wrong return type is intercepted we need to throw an exception. Hence above examples will throw an exception.

Possible extension

One possible extension would be do declare a restriction on what to intercept like

doReturn("mocked return value").when(MyOtherClass.class, () -> {
    MyOtherClass o = MyClass.firstMethod();
    return o.secondMethod()
});

Which declares that we want to set up mocking for MyOtherClass, not MyClass.

Conflict between old and new doReturn-when

doReturn(() -> "A").when(() -> MyClass.firstMethod());

In above case Mockito needs to decide if this is the new setup method or an incomplete version of

Supplier<String> m = mock(Supplier.class);

doReturn(() -> "A").when(m).get();

it can do this by checking if the parameter to m is a mock during runtime, but the when(λambda) API needs to support both cases, hence when(λambda) has to return a lambda type, i.e.

LambdaStubber<Supplier<T>> doReturn(Supplier<T> ret) {
    return new LambdaStubber<Supplier<T>>(ret);
}

public static class LambdaStubber<T> implements Stubber {
    Supplier<T> when(Supplier<T>) {
        ....
    }
}
Possible extension

This issue might be mitigated by introducing

doReturn(() -> "A").whenStatic(() -> MyClass.firstMethod());

T whenStatic(Class<T>)

doReturn("mocked return value").whenStatic(MyClass.class).myStaticMethod();

verifyStatic(MyClass.class).myStaticMethod();

doReturn(...).whenStatic(MyClass.class) returns a mock instance of MyClass with all instance methods throwing a stubbing exception. So the user will not accidentally call a instance method on this special stubbing-mock.

Problems

doReturn(...).whenStatic(MyClass.class) returns a mock instance of MyClass. Hence calling doReturn(...).whenStatic(MyClass.class).myStaticMethod() is equivalent to

MyClass m = mock(Myclass.class);
m.myStaticMethod();

While it is valid Java to call static methods on instances, all IDE code checkers, linters and tools like errorprone will light up like a christmas tree and tell you not to do it.

Life-time of static mocking

Regular instance mocking is tied to an object's lifetime. Static methods do not have such a life-time, hence when to know when to reset the methods and stop collecting invocations.

mockStatic(Class) -> finishStaticMocking(Class)

This is the most basic solution. The advantage is that this works with every possible test-framework and if someone wants to have a nicer API they can e.g. build a AutoClosable that automatically calls finishStaticMocking(...) when a try-with-resources block is exited.

MockitoSessions

As far as I understood there can only be one session at a time. Hence scenarios with interleaved life-cycles of static mocking can not be supported. Example:

mockStatic(A.class)
mockStatic(B.class)

when(A.m()).thenReturn("mockA")
when(B.m()).thenReturn("mockB")

assertEquals("mockA", A.m());

finishStaticMocking(A.class)

assertEquals("mockB", B.m());

finishStaticMocking(A.class)

Of course if the user wants to use sessions (or other session like tools, such as the Mockito JUnit runner) they should apply to static mocking too.

@avanathan

This comment has been minimized.

Show comment
Hide comment
@avanathan

avanathan Jul 30, 2018

@moltmann Could you please open PR for your changes?

avanathan commented Jul 30, 2018

@moltmann Could you please open PR for your changes?

@moltmann

This comment has been minimized.

Show comment
Hide comment
@moltmann

moltmann Jul 30, 2018

Contributor

@avanathan My code is currently based on dexmaker-mockito and does not work with the bytebuddy mock maker used by vanilla Mockito. I think we should first agree on the interface before trying to make this work with bytebuddy.

Contributor

moltmann commented Jul 30, 2018

@avanathan My code is currently based on dexmaker-mockito and does not work with the bytebuddy mock maker used by vanilla Mockito. I think we should first agree on the interface before trying to make this work with bytebuddy.

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Jul 31, 2018

Member

@moltmann, thank you for working on this! I plan to re-engage soon.

Member

mockitoguy commented Jul 31, 2018

@moltmann, thank you for working on this! I plan to re-engage soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment