Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1. bulk_mode. 2. _execute_step. 3. Type checking. #1632

Merged
merged 4 commits into from
Mar 24, 2020

Conversation

prescod
Copy link
Contributor

@prescod prescod commented Mar 22, 2020

  1. I forgot bulk_mode in the schema because there were no example mapping files with it. Fixed now.

  2. Renamed _load_mapping to _execute_step because _load_mapping always implied to me that it was loading the mapping file. This is subjective so tell me if you disagree.

  3. I added stronger type checking which is how I found problem 1. Integration tests don't use bulk_mode but unit tests do

  4. I deleted the test that checks what happens if a bulk_mode is set to a wrong value in mapping because the mapping parser should handle that now and it is hard to trigger with both pytest and typeguard working at the same time. In any case, the Salesforce API would give you an error pretty quickly if you found a way around all of this type checking...

1. I forgot bulk_mode in the schema because there were no example mapping files with it. Fixed now.

2. Renamed _load_mapping to _execute_step because _load_mapping always implied to me that it was loading the mapping file.

3. I added stronger type checking which is how I found problem 1. Integration tests don't use bulk_mode but unit tests do.
def _load_mapping(self, mapping):
def _execute_step(
self, mapping: MappingStep
) -> Union[DataOperationJobResult, MagicMock]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like having to include mocks in the type signature. Is there no way to tell pydantic to always allow them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mean typeguard.

And yes, there is a plan to fix it at the Typeguard level but not merged yet.

agronholm/typeguard#105

I'll bug the maintainer again, but if he doesn't respond we might have to figure out whether to fork, abandon or continue to work around small bugs in typeguard. I found another one on the weekend. Easy to fix, but only if he accepts my PRs.

@davisagli
Copy link
Contributor

@prescod the tests are failing because this imports from typing_extensions which is not in requirements

@prescod prescod requested a review from davisagli March 23, 2020 21:40
@davisagli davisagli merged commit 3ed1357 into master Mar 24, 2020
@davisagli davisagli deleted the feature/load_data_enhancements branch March 24, 2020 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants