Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR: Implement support for better ndarray copy management in colour.continuous.Signal class. #1151

Merged
merged 2 commits into from May 16, 2023

Conversation

KelSolaar
Copy link
Member

Summary

This PR implements support for better ndarray copy management in colour.continuous.Signal class enabling most internal operations to be performed without a copy of the data and thus enabling better performance.

The newly added colour.utilities.ndarray_copy_enable context manager / decorator should also be useful for other areas touched by #1120. I purposely left them untouched in this particular PR.

Preflight

Code Style and Quality

  • Unit tests have been implemented and passed.
  • Pyright static checking has been run and passed.
  • Pre-commit hooks have been run and passed.
  • [N/A] New transformations have been added to the Automatic Colour Conversion Graph.
  • [N/A] New transformations have been exported to the relevant namespaces, e.g. colour, colour.models.

Documentation

  • New features are documented along with examples if relevant.
  • The documentation is Sphinx and numpydoc compliant.

@coveralls
Copy link

coveralls commented May 13, 2023

Coverage Status

Coverage: 99.785% (+0.0003%) from 99.784% when pulling 79bc5ba on feature/copyless into 96d9c5d on develop.

Copy link
Contributor

@tjdcs tjdcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing changes requested.

with ndarray_copy_enable(False):
self.assertEqual(id(ndarray_copy(a)), id(a))


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should test the other decorating modes as well. This leaves the function decorator untested.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -1788,6 +1792,142 @@ def from_range_int(
return a


_NDARRAY_COPY_ENABLED: bool = True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is something we should do (long term, not this PR) to manage the number of globals Colour creates. A lot of the caches are also globals. Users might not want to see these globals if they are trying to debug client applications, or at least it could be clearer that they are colour-"globals"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point yeah, it should be easy to do!

Copy link
Contributor

@tjdcs tjdcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing changes requested.

@@ -515,6 +517,7 @@ def extrapolator_kwargs(self, value: dict):
self._function = None # Invalidate the underlying continuous function.

@property
@ndarray_copy_enable(False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me why these decorators are littered around... I can't see where they are actually affecting any behavior. It looks like it is a promise that this function is copy-less safe (no-side effects) but that guarantee or promise didn't exist before and doesn't appear to be adding value to the function signature here... Are we going to end up with @nd_copy_enable(False) functions everywhere?

Minor concern about long term code litter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good concern, and one would certainly have to know what is going on here, i.e. that it disable the domain and range getter copies. We could add some comments to make it understandable. I don't think there is a need to have the decorators anywhere else than where they are.

An alternative is to have getters that are without copy, e.g. domain_no_copy or alike.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did a small pass using directly the protected attributes thus removed most of the decorated calls.

Copy link
Contributor

@tjdcs tjdcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ready to me! Testing went well.

Will refactor performance branch on this... eventually.

@KelSolaar KelSolaar merged commit 93deeab into develop May 16, 2023
25 of 26 checks passed
@KelSolaar KelSolaar added this to the v0.4.3 milestone May 16, 2023
@tjdcs tjdcs deleted the feature/copyless branch June 29, 2023 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants