Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not serialize all attributes for updates and deletes #905

Merged
merged 6 commits into from Feb 12, 2021

Conversation

ikonst
Copy link
Contributor

@ikonst ikonst commented Feb 11, 2021

Not only it is more efficient not to serialize attributes you don't need, it also manifested in an issue where an update operation would fail null-checks on attributes that are not even affected by the update -- see added test_update_doesnt_do_validation_on_null_attributes test.

p.s. this uncovered another bug where we fail to propagate null_checks=True through MapAttributes, but we'll address it separately

@@ -474,34 +461,51 @@ def refresh(self, consistent_read: bool = False, settings: OperationSettings = O
raise ValueError("Cannot refresh this item from the returned class: {}".format(stored_cls.__name__))
self.deserialize(item_data)

def get_operation_kwargs_from_instance(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break into 3 variants even at the cost of slight repetition

@@ -928,16 +932,14 @@ def _get_indexes(cls):
cls._indexes['local_secondary_indexes'].append(idx)
return cls._indexes

def _get_save_args(self, attributes=True, null_check=True):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The attributes flag is no longer needed (always True).

return args, kwargs

def _handle_version_attribute(self, serialized_attributes, actions=None):
def _get_hash_range_key_serialized_values(self) -> Tuple[Any, Optional[Any]]:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method replaces _get_save_args for non-save scenarios by serializing only the HK and RK.

if not TestModel.exists():
print("Creating table")
TestModel.create_table(read_capacity_units=1, write_capacity_units=1, wait=True)
if TestModel.exists():
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean table between tests

@ikonst ikonst changed the title Stop using _get_save_args for non-saves Do not serialize all attributes for updates and deletes Feb 11, 2021
elif 'range_key' in save_kwargs:
kwargs['range_key'] = save_kwargs['range_key']
return self._get_connection().get_operation_kwargs(*args, **kwargs)
return self._get_connection().get_operation_kwargs(hk_value, range_key=rk_value, key=KEY, actions=actions, condition=condition, return_values_on_condition_failure=return_values_on_condition_failure)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've really been meaning to setup black to lint 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I'll withhold formatting cause there's no point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly

def _handle_version_attribute(self, serialized_attributes, actions=None):
def _get_hash_range_key_serialized_values(self) -> Tuple[Any, Optional[Any]]:
if self._hash_keyname is None:
raise Exception("The model has no hash key")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like precedent is a ValueError -

raise ValueError(f"This model has no hash key, but a hash key value was provided: {range_key}")

Copy link
Contributor Author

@ikonst ikonst Feb 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considered that, but in that precedent it actually relates to an argument that was passed in.

also lol {range_key}

@ikonst
Copy link
Contributor Author

ikonst commented Feb 12, 2021

🚀

@ikonst ikonst merged commit 6a75cf4 into pynamodb:master Feb 12, 2021
@ikonst ikonst deleted the save_args branch February 12, 2021 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants