Skip to content
This repository was archived by the owner on Nov 17, 2025. It is now read-only.

Fix subtensor shape inference #39

Merged
merged 22 commits into from
Oct 12, 2020
Merged

Fix subtensor shape inference #39

merged 22 commits into from
Oct 12, 2020

Conversation

@brandonwillard
Copy link
Member

@brandonwillard brandonwillard commented Sep 16, 2020

This PR adds generic shape inference for basic and advanced NumPy indexing on Theano tensors.

Note: the boolean case isn't covered, yet.

@brandonwillard brandonwillard self-assigned this Sep 16, 2020
@brandonwillard brandonwillard marked this pull request as draft September 16, 2020 20:14
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch 9 times, most recently from 7da5dc8 to 82dc232 Compare September 21, 2020 02:45
@brandonwillard brandonwillard linked an issue Sep 26, 2020 that may be closed by this pull request
Fix constant folding optimization error caused by AdvancedBooleanSubtensor #34
Closed
@brandonwillard
Copy link
Member Author

brandonwillard commented Sep 26, 2020 •
edited
Loading

Currently, I'm stuck considering whether or not we should attempt to use tt.nonzero to convert boolean indices into "regular" non-boolean indices. This seems to work rather well and cuts down the amount of boolean-specific logic need in the *Subtensor* Ops (it also entirely obviates some Subtensor Ops); however, it has one distinct disadvantage: it doesn't produce the same (runtime) errors as NumPy.

This can arise when nonzero reduces the dimension size of the boolean indices so that they actually match an indexed array with smaller dimensions. When this happens, the Op.perform will succeed without error, yet the corresponding NumPy operation would fail.

We could use an Assert node that would raise an exception when the dimensions of the boolean indices and the indexed array do not match, but, unfortunately, this exception wouldn't match NumPy's exception type. Also, Assert nodes can be removed during certain forms of optimization, and we wouldn't ever want to remove these ones.

@twiecki
Copy link
Contributor

twiecki commented Sep 26, 2020

I don't see the numpy-incompatibility here as a blocker and the assert node (even with a different exception) is a reasonable solution.

@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch 4 times, most recently from 213369f to 09897cc Compare October 5, 2020 03:17
@brandonwillard brandonwillard removed a link to an issue Oct 5, 2020
Fix constant folding optimization error caused by AdvancedBooleanSubtensor #34
Closed
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch from 09897cc to ac5a8b5 Compare October 5, 2020 06:22
@brandonwillard
Add a Makefile
a9e8c7c
@brandonwillard
Update logger library usage
e842fd1
@brandonwillard
Misc. style and format updates
f14d479
@brandonwillard
Move merge-based graph functions and equal_computations
1ff98d2 
The merge-based functions were moved to `toolbox` to reduce unnecessary
cross-module dependencies (especially among core modules).  `equal_computations`
was moved from `scan_module` because it provides a basic graph object `__eq__`
implementation and it has multiple references outside of its own
module/sub-package.
@brandonwillard
Use original traceback when raising shape errors
6d55c43
@brandonwillard
Raise shape and optimization errors by default during testing
0e0f613
@brandonwillard
Make get_scalar_constant_value compatible with NoneTypes
adfeaaf
@brandonwillard
Add a broadcast_shape function
b210efb
@brandonwillard
Add general shape inference for all types of non-boolean indexing
73d798a
@brandonwillard
Remove unnecessary Subtensor dependencies in nonzero functions
8036b14
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch from ac5a8b5 to 9d251af Compare October 11, 2020 06:47
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch from 9d251af to 9895ef3 Compare October 11, 2020 06:52
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch 6 times, most recently from 8a7e2ce to 5bc5198 Compare October 11, 2020 22:43
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch from 5bc5198 to 710b3ee Compare October 11, 2020 22:50
@brandonwillard
Make as_tensor_variable's handling of constants consistent
24b8bc2 
Now, when `as_tensor_variable` is given `Constant` types, it will return the
same result as a call with the underlying non-`Variable` data.

Closes Theano#98
@brandonwillard
Use change_flags instead of try...finally statements in test_compute_…
eaf1470 
…test_value.py
@brandonwillard brandonwillard force-pushed the fix-subtensor-shape-inference branch from 710b3ee to eaf1470 Compare October 12, 2020 00:44
@brandonwillard
Copy link
Member Author

brandonwillard commented Oct 12, 2020

This PR has grown to include quite a few independently important changes that were needed along the way—as well as some helpful refactoring. Due to the general importance of these changes, I'm going to consider this PR finished once the tests pass and continue the boolean indexing changes in another PR.

@brandonwillard brandonwillard marked this pull request as ready for review October 12, 2020 01:21
@brandonwillard
Copy link
Member Author

brandonwillard commented Oct 12, 2020

Well, this recent run included the removal of constant caching (i.e. #99), and I don't see any noticeable changes in the speed of the tests. The claim was that that it would speed up the MergeOptimizer (with no measures attached to said claim), so—worst case scenario—I was expecting a slowdown that would at least be noticeable in the aggregate. Alas, I've observed no such slowdown in the tests.

Locally, I'm also not seeing any real changes in memory use. It obviously wouldn't hurt to directly profile this change, but, given the simplicity now provided by the removal of constant caching, I believe we should move forward with this change.

@brandonwillard brandonwillard merged commit e96b285 into aesara-devs:master Oct 12, 2020
@brandonwillard brandonwillard deleted the fix-subtensor-shape-inference branch October 12, 2020 02:42
@brandonwillard brandonwillard mentioned this pull request Oct 14, 2020
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

@brandonwillard brandonwillard

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@brandonwillard @twiecki