[MAINTENANCE] Small refactor of ExecutionEngine.resolve_metrics() for better code readability (and miscellaneous additional clean up) #6587
Conversation
✅ Deploy Preview for niobium-lead-7998 ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
alexsherstinsky
changed the title
👇 Click on the image for a new way to code review
Legend |
| @@ -98,7 +98,7 @@ def active_batch_id(self) -> Optional[str]: | |||
| the list of "Batch" objects, each of whose BatchData has already been loaded (and cached). Since BatchData IDs | |||
| are from the same name space as Batch IDs, this helps avoid unnecessary loading of data from different backends. | |||
| """ | |||
| active_batch_data_id: Optional[str] = self._active_batch_data_id | |||
| active_batch_data_id: Optional[str] = self.active_batch_data_id | |||
There was a problem hiding this comment.
Was this a bug? What is the difference between these two attributes?
There was a problem hiding this comment.
@NathanFarmer Using full property takes advantage of the checks that property method performs -- this is more robust.
| metric_fn_bundle_configurations, | ||
| ) | ||
|
|
||
| if metrics is None: |
There was a problem hiding this comment.
Probably want to keep with the pattern used elsewhere: if not metrics
There was a problem hiding this comment.
@NathanFarmer In this case we are actually focusing on "None" (as opposed to "having nothing in it" -- i.e., being "Falsy"). So this is a stronger statement. Happy to discuss. Thanks!
…erstinsky/batch_metric_store/batch_manager/metrics/execution_engine/cleanup_and_resolve_metrics_code_simplification-2022_12_15-280
| @@ -444,11 +444,13 @@ def _get_reader_fn(self, reader_method=None, path=None): | |||
| f'Unable to find reader_method "{reader_method}" in pandas.' | |||
| ) | |||
|
|
|||
| def resolve_metric_bundle( # type: ignore[empty-body] | |||
| def resolve_metric_bundle( | |||
| self, metric_fn_bundle | |||
There was a problem hiding this comment.
@alexsherstinsky can you add an annotation (other than Any) for metric_fn_bundle?
There was a problem hiding this comment.
@Kilo59 No -- this has already been discussed before -- it can be Callable, any of a great many of SqlAlchemy functions, and any of a great many Spark functions. I do not see an easy possibility here. Happy to discuss. Thanks.
There was a problem hiding this comment.
Callable is better than no annotation.
alexsherstinsky
deleted the
maintenance/GREAT-1347/GREAT-181/GREAT-1445/alexsherstinsky/batch_metric_store/batch_manager/metrics/execution_engine/cleanup_and_resolve_metrics_code_simplification-2022_12_15-280
branch
* develop: [FEATURE] Ensure `result_format` accessed is through Checkpoint, and warns users if `Expectation` or `Validator`-level (#6562) [BUGFIX] Remove rendered header from Cloud-rendering tests (#6597) [MAINTENANCE] Refactor `BaseDataContext` and `DataContext` into factory functions (#6531) [MAINTENANCE] Utilize a `StrEnum` for `ConfigPeer` modes (#6596) [BUGFIX] Use v3.3.6 or higher of google-cloud-bigquery (with shapely bugfix) (#6590) [MAINTENANCE] Add docs snippet checker to `dev` CI (#6594) [MAINTENANCE] Leverage `RendererConfiguration` in existing prescriptive templates (3 of 3) (#6530) [BUGFIX] Support non-string `datetime` evaluation parameters (#6571) [RELEASE] 0.15.41 (#6593) [FEATURE] ZEP - PG `BatchSorter` loading + dumping (#6580) [MAINTENANCE] Leverage `RendererConfiguration` in existing prescriptive templates (2 of 3) (#6488) [MAINTENANCE] Remove `ExplorerDataContext` (#6592) [MAINTENANCE] Small refactor of ExecutionEngine.resolve_metrics() for better code readability (and miscellaneous additional clean up) (#6587) [MAINTENANCE] `mypy` config update (#6589) [BUGFIX] convert_to_json_serializable does not accept numpy datetime (#6553) [BUGFIX] Return unique list of batch_definitions (#6579) [MAINTENANCE] typo in method name (#6585)
Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], or [CONTRIB]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.
Changes proposed in this pull request:
After submitting your PR, CI checks will run and @cla-bot will check for your CLA signature.
For a PR with nontrivial changes, we review with both design-centric and code-centric lenses.
In a design review, we aim to ensure that the PR is consistent with our relationship to the open source community, with our software architecture and abstractions, and with our users' needs and expectations. That review often starts well before a PR, for example in GitHub issues or Slack, so please link to relevant conversations in notes below to help reviewers understand and approve your PR more quickly (e.g.
closes #123).Previous Design Review notes:
Definition of Done
Please delete options that are not relevant.
Thank you for submitting!