-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MAINTENANCE] Add unit test for MetricsCalculator.get_metric() Method -- as an example template #6179
[MAINTENANCE] Add unit test for MetricsCalculator.get_metric() Method -- as an example template #6179
Conversation
…YYYY/alexsherstinsky/validator/metrics_calculator/add_unit_test_for_get_metric-2022_10_19-249
✅ Deploy Preview for niobium-lead-7998 ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
👇 Click on the image for a new way to code review
Legend |
…test_metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's very unclear what the value or intent of this test is.
I would suggest checking the actual code coverage of this test and make sure it's exercising the sections of the code you intend.
From what I can see, you don't need to use mock.patch
at all.
@mock.patch("great_expectations.execution_engine.execution_engine.ExecutionEngine") | ||
def test_get_metric(execution_engine: mock.MagicMock): | ||
metrics_calculator = MetricsCalculator(execution_engine=execution_engine) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should (almost) never use MagicMock
without providing a spec.
https://realpython.com/python-mock-library/#avoiding-common-problems-using-specifications
An Even better alternative would be to define a proper double and just pass in the object to MetricsCalculator(execution_engine=exec_engine_double)
. Then we have proper isolation and a clear contract, and then we know for certain that a change to ExecutionEngine
will never break this test.
class ExecutionEngineStub:
def some_method_the_test_needs(self) -> str:
return "foobar"
def test_get_metric_does_something_specific():
metrics_calculator = MetricsCalculator(execution_engine=ExecutionEngineStub())
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Kilo59 We are not using any methods inside ExecutionEngine
here -- it is only present as an argument for instantiating MetricsCalculator
. Does this change your recommendation? Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it just means the double can be very simple (AKA a dummy). As in, it doesn't have to do anything.
It might even be None
.
class ExecutionEngineDummy:
pass
def test_get_metric_does_something_specific():
metrics_calculator = MetricsCalculator(execution_engine=ExecutionEngineDummy())
...
metrics_calculator = MetricsCalculator(execution_engine=object())
...
metrics_calculator = MetricsCalculator(execution_engine=None)
"great_expectations.validator.metric_configuration.MetricConfiguration.metric_name", | ||
new_callable=mock.PropertyMock, | ||
return_value=metric_name, | ||
), mock.patch( | ||
"great_expectations.validator.metric_configuration.MetricConfiguration.metric_domain_kwargs", | ||
new_callable=mock.PropertyMock, | ||
return_value=IDDict(metric_domain_kwargs), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like you should be using a MetricConfiguationStub
here (or the real thing) instead patching in this stuff.
Makes the test much more confusing then it needs to be.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Kilo59 Let us discuss to make sure we agree on the best alternative. Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with what you have now, but if you want isolation from the real MetricConfiguation
you should be able to just use a namedtuple
(or IDDict
/DotDict
) in place of the real thing.
I still don't see the need for the mock.patch
here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Kilo59 The MetricConfiguration
object is the one input to get_metric()
-- the method under test. So we need it, but its internals must be maximally mocked leaving only the aspects necessary for the test as un-mocked (to prevent test leakage). I updated the docstring to emphasize this. Thanks!
…test_metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247
…metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247' of https://github.com/great-expectations/great_expectations into maintenance/GREAT-1103/GREAT-1316/alexsherstinsky/test_metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247
…rstinsky/test_metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247
…rstinsky/test_metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247
e323798
to
fedd399
Compare
…rstinsky/test_metrics_calculator/test_get_metric_unit_test_patched_with_mocks-2022_10_19-247
3c25be7
to
b7367e2
Compare
Scope
A tight unit test for the
MetricsCalculator.get_metric(MetricConfiguration)
method is sought, such that the implementation of this test can be used as an example for developing other tight unit tests.General Approach
Use the “Mock” approach for as many boundary objects/methods surrounding the core functionality of the method under test.
Particular Approach
MetricsCalculator
is under test andMetricsCalculator
is instantiated withExecutionEngine
, use amock.MagicMock
for theExecutionEngine
constructor argument.MetricsCalculator.get_metric(MetricConfiguration)
consists of callingMetricsCalculator.get_metrics(Dict[str, MetricConfiguration])
and extracting the key corresponding tometric_name
, usemock.patch
for theMetricsCalculator.get_metrics
return value.MetricConfiguration
is not under test, use mock.patch for theMetricConfiguration.metric_name
andMetricConfiguration.metric_domain_kwargs
properties, respectively.MetricConfiguration
constructor arguments and the mock.patch return values for the corresponding properties and method return values are mutually consistent.Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], or [MAINTENANCE]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.
Changes proposed in this pull request:
After submitting your PR, CI checks will run and @cla-bot will check for your CLA signature.
For a PR with nontrivial changes, we review with both design-centric and code-centric lenses.
In a design review, we aim to ensure that the PR is consistent with our relationship to the open source community, with our software architecture and abstractions, and with our users' needs and expectations. That review often starts well before a PR, for example in github issues or slack, so please link to relevant conversations in notes below to help reviewers understand and approve your PR more quickly (e.g.
closes #123
).Previous Design Review notes:
Definition of Done
Please delete options that are not relevant.
Thank you for submitting!