Adding convolutional version of MNIST example?

[capn-freako]

Hi Lars,

Thanks for this library!

After studying the code in your MNIST.hs example, I assume that you are not, yet, using convolutional layers in your MNIST digit classification example. Is that correct? If so, I'd like to contribute to this project, by adding that (i.e. - modifying the MNIST.hs example to use convolutional layers).

Did you already have an idea in mind, as to how you'd like to proceed with this?

And, finally, I was hoping you might have time to consider these two questions:

1. In MNIST.hs, why is the MNISTModel type defined as:

   type MNISTModel = Classifier (Matrix 28 28) 10 Img Digit

   when the learning appears to work on the flattened 784-element vector, instead?
   This choice seems to force mnistModel.f to come inside the learning loop and, therefore, be defined as a Diff, whereas otherwise it might have been defined as a simple function (just flattening the 2D image into a 1D vector, and remaining outside of the learning loop).

2. Referring to this comment:

   A Classifier f n b c is a Model that classifies items of type b into categories of type c, using a component with input shape f and output shape Vector n.

   Why is the apparent type redundancy required?
   In other words, why not just Classifier f n (or, Classifier b c)?

Thanks!
-db

[capn-freako]

Hi Lars,

Can you tell me if these changes to your MNIST.hs file indicate that I correctly understand your intent, regarding the use of type annotations to control the structure of the resultant network?

-    c = tanhLayer' . (tanhLayer' :: Layer 784 30) . cArr f
+    c = reLULayer
+      . cArr (Diff toVector)
+      . (convolution (Proxy :: DP.Proxy 7) 3 linearLayer :: Component (Volume 28 28 1) (Volume 8 8 8))
+      . cArr (Diff fromMatrix)

Thanks,
-db

[brunjlar]

Hi David, thank you very much for your interest in my library! I am very sorry for the late reply - I am quite busy at work these days. It would be great to use convolutional layers for the MNIST example, and I'd be really grateful if you could do that! Regarding your questions: 1. You are right - I think I defined the type like this because it is "nicer" and more intuitive (an image *is* two-dimensional after all), but performance probably suffers for it. Would be nice if things like this didn't have an impact on performance... 2. Well, Classifier f n would be quite "low level", because you'd still need to decode the resulting number into the type c that you're actually interested in. For your second option, Classifier b c, one would need a way to "know" that c has n elements on the type level, wouldn't one? If possible somehow, the second option would indeed by nicer. Greetings - Lars

…

On Wed, Oct 4, 2017 at 8:22 PM, David Banas ***@***.***> wrote: Hi Lars, Thanks for this library! After studying the code in your *MNIST.hs* example, I assume that you are not, yet, using convolutional layers in your MNIST digit classification example. Is that correct? If so, I'd like to contribute to this project, by adding that (i.e. - modifying the MNIST.hs example to use convolutional layers). Did you already have an idea in mind, as to how you'd like to proceed with this? And, finally, I was hoping you might have time to consider these two questions: 1. In *MNIST.hs*, why is the *MNISTModel* type defined as: type MNISTModel = Classifier (Matrix 28 28) 10 Img Digit when the learning appears to work on the flattened 784-element vector, instead? This choice seems to force *mnistModel.f* to come inside the learning loop and, therefore, be defined as a *Diff*, whereas otherwise it might have been defined as a simple function (just flattening the 2D image into a 1D vector, and remaining outside of the learning loop). 2. Referring to this comment: A Classifier f n b c is a Model that classifies items of type b into categories of type c, using a component with input shape f and output shape Vector n. Why is the apparent type redundancy required? In other words, why not just Classifier f n (or, Classifier b c)? Thanks! -db — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGA832nSg2TRTZpc_bWtI3G1g1pKoV59ks5so8zZgaJpZM4PuCHM> .

[brunjlar]

Hi David, yes, looks good. It was indeed my intent to use type annotations. (Maybe nowadays, this could be done in a nicer way using the TypeApplications extension...) Greetings - Lars

…

On Thu, Oct 5, 2017 at 2:58 AM, David Banas ***@***.***> wrote: Hi Lars, Can you tell me if these changes to your *MNIST.hs* file indicate that I correctly understand your intent, regarding the use of type annotations to control the structure of the resultant network? - c = tanhLayer' . (tanhLayer' :: Layer 784 30) . cArr f+ c = reLULayer+ . cArr (Diff toVector)+ . (convolution (Proxy :: DP.Proxy 7) 3 linearLayer :: Component (Volume 28 28 1) (Volume 8 8 8))+ . cArr (Diff fromMatrix) Thanks, -db — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGA835uNo2wKvB5k-cX59DS4aNDvJaJXks5spCm1gaJpZM4PuCHM> .

[capn-freako]

Hi Lars, Thanks for all your replies!

…

-db

On Oct 15, 2017, at 2:48 AM, Lars Brünjes ***@***.***> wrote: Hi David, yes, looks good. It was indeed my intent to use type annotations. (Maybe nowadays, this could be done in a nicer way using the TypeApplications extension...) Greetings - Lars On Thu, Oct 5, 2017 at 2:58 AM, David Banas ***@***.***> wrote: > Hi Lars, > > Can you tell me if these changes to your *MNIST.hs* file indicate that I > correctly understand your intent, regarding the use of type annotations to > control the structure of the resultant network? > > - c = tanhLayer' . (tanhLayer' :: Layer 784 30) . cArr f+ c = reLULayer+ . cArr (Diff toVector)+ . (convolution (Proxy :: DP.Proxy 7) 3 linearLayer :: Component (Volume 28 28 1) (Volume 8 8 8))+ . cArr (Diff fromMatrix) > > Thanks, > -db > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#5 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AGA835uNo2wKvB5k-cX59DS4aNDvJaJXks5spCm1gaJpZM4PuCHM> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA_7mq4awQfNzf-65cX-YcRHjmv8lxU-ks5ssdT6gaJpZM4PuCHM>.