Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Breaking] Adding dilation/groups to Conv2D. Adding dilation to Pool2D #767

Merged
merged 23 commits into from
May 8, 2023

Conversation

coreylowman
Copy link
Owner

@coreylowman coreylowman commented May 5, 2023

Resolves #693
Resolves #589

Breaking API changes

  • Two new parameters are added to Conv2D and the Min/Max/Avg Pool2D nn layers
  • API for tensor op conv2d is now implemented on a tuple of the image & weight, instead of accepting filters as a parameter. Additionally, the stride/padding/dilations/groups are now passed into the method instead of being const parameters. A call to conv2d with default parameters for things would look like this:
(img, filters).conv2d(Const::<1>, Const<0>, Const::<1>, Const::<1>);
  • Removed min_pool2d/max_pool2d/avg_pool2d
  • TryPool2D now accepts the type of pooling/kernel/stride/padding/dilation as parameters:
img.pool2d(Pool2DKind::Min, Const::<2>, Const::<1>, Const::<0>, Const::<1>);

New features

  • Added dilation parameter to conv2d
  • Added group parameter to conv2d
  • Added dilation parameter to pool2d
  • All dimensions of image/filter tensors support usize now (previously channel was const)
  • Stride/padding/dilation/group can now be usize values if the tensor dimensions are usize

Tests ran

  • cargo +nightly test --tests
  • cargo +nightly test --tests -F test-f64
  • cargo +nightly test --tests -F test-f16
  • cargo +nightly test --tests -F cuda
  • cargo +nightly test --tests -F cuda,test-f64
  • cargo +nightly test --tests -F cuda,test-f16
  • cargo +nightly test --tests -F cudnn
  • cargo +nightly test --tests -F cudnn,test-f64
  • cargo +nightly test --tests -F cudnn,test-f16

@coreylowman coreylowman changed the title [WIP] Adding dilation/group parameter to Conv2d [WIP] [Breaking] Adding dilation/group parameter to Conv2d May 5, 2023
@coreylowman coreylowman added the breaking-change Semver breaking change label May 5, 2023
@coreylowman coreylowman changed the title [WIP] [Breaking] Adding dilation/group parameter to Conv2d [Breaking] Adding dilation/groups to Conv2D. Adding dilation to Pool2D May 8, 2023
@coreylowman coreylowman merged commit b7d95e2 into main May 8, 2023
8 checks passed
@coreylowman coreylowman deleted the conv-upgrades branch May 8, 2023 16:36
mattjurenka pushed a commit to mattjurenka/dfdx that referenced this pull request May 13, 2023
coreylowman#767)

* Forward pass for CPU working with dilation/groups

* Adding test generation script

* Backward img grad working

* Backward filters working

* Removing large test case

* Adding sketches of cuda/cudnn kernels

* Update cudnn code

* Temp commit

* Update filters transpose logic for cuda

* Cuda kernel implementation

* batched conv2d tests

* Fixing dilation

* Cudnn passing tests

* cuda tests passing

* Reverting numpy tests

* Removing old conv2d implementation

* Pool2D rewrite

* Updates for pool2d cuda

* Adding dilated pool2d test

* Adding dilation & groups to nn layer

* Adding conv2d dilation & group tests

* Fixing tests & warning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Semver breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dilation for Conv2d and MaxPool2d ? Add Conv2D groups
1 participant