What needs work on?

[Jxrgxn]

Hi. Since Github eliminated private messaging I find it difficult to comunicate with developers. I'm just wondering if there's a feature I can tackle or code I can take off yoru plate.

[Jxrgxn]

Nevermind, with your blessing @collinhundley I'd like to take on support for multiple hidden layers / and deep learning.

[collinhundley]

Yes, that would be great if you'd like to work on multiple hidden layers for FFNN. I'd recommend waiting a few hours though - @perlfly has made some big changes to help support deep learning, and I'll hopefully have time to review and merge it soon!

[perlfly]

Great @KacheFlowe ! I'd appreciate your comments on my latest PR. It adds a struct that could encapsulate the layer logic. At the moment it support forward calculation. It lacks backward propagation and Softmax-ing. Feel free to ask any clarification where my code is not clear enough.

If @collinhundley agrees with the direction taken by my pr, I propose opening a dedicated FFNN branch where you could contribute directly.

[Jxrgxn]

@perlfly Yeah sure. Are you a native Swifter? P.S. If I get anything wrong, please forgive me. And I hope in my comments I do not come off as arrogrant or abrasive. Please let me know if my tone is inappropriate.

[Jxrgxn]

@collinhundley Hey Colin. I noticed you merged the branch and in your comments mentioned that you'll need to look at the code later. I can help you look at the code if you'd like. And I can begin work whenever.

[collinhundley]

I've merged PR #23. This will allow everybody to continue with @perlfly 's method of handling multiple hidden layers and support for deep learning.

Please keep performance in mind when adding any new code. Although readability is an important characteristic, for this particular library I would like to make performance an even higher priority. For large neural networks, even microseconds can add up very quickly so it's important that we keep the calculations as efficient as possible. For this reason, we need excellent documentation to keep the code maintainable. Don't be afraid of excessive comments - make note of anything that might help another developer understand your logic.

If anybody is looking for something to tackle, I would love to see Softmax activation working. I began implementing it, but got a little hung up with applying its derivative during backpropagation. See the FIXME under 'Calculate output errors' in the backpropagate method. Currently Softmax normalizes the outputs correctly, but it always converges on a wrong answer, which suggests that it's probably a backpropagation issue. Ultimately it should be used for the 'Handwriting' example (trainer in the OS X app, classifier in the iOS app).

[Jxrgxn]

@collinhundley I'm pulling @perlfly changes to my fork now and will then begin work on Softmax activation. Also I document excessively under normal circumstances so that isn't an issue.

[collinhundley]

@KacheFlowe Thanks, I should have more time to look the changes tomorrow (U.S. Mountain Time). And @perlfly I appreciate your work, it looks great.

[Jxrgxn]

@collinhundley Would you prefer that I contribute to an FFNN branch directly or submit pull requests from my own repo?

[collinhundley]

@KacheFlowe Go ahead and submit pull requests from your own repo for now. We can add another branch later if it's more convenient.

[Jxrgxn]

@collinhundley rubs hands together Oh yeaaaah. Here we go

[perlfly]

@KacheFlowe Please make sure you do not look at FFNN branch in my fork (on my git account). It was just something like a playground to test different direction. FFNN_Prop (proposal) is what I proposed.

Speaking of Softmax, I was wondering if it is the case to add that logic to the Layer or to the ActFinction.
At the moment the previous Softmax logic is commented inside the Layer struct.

BTW: What is a native swifter? :-) ... Oh yes ... I have 5+ years of Swift experience ;-)

[Jxrgxn]

@perlfly haha I'm sorry! I meant to write native iOS Developer. Sorry. I hadn't slept for a whole day. Not that it matters ultimately, I was simply curious :)

[perlfly]

Yes @KacheFlowe I developed a few small iOS apps since iOS5, but I'm primarily focused on the language itself (Swift) as a developer skill and Machine Learning. So this project fits perfectly with my interests.

[Jxrgxn]

@perlfly Ah! Looks like we have the same interests then 👍

[Jxrgxn]

@collinhundley So I saw a post a couple of days ago detailing that you created another branch. Will I be added toa collaborater so that I may work on the branch directly (and thus create sub branches?). I won't object to this system if it is what you desire but it seems a bit cumbersome from a contributor's perspective. It'd be easier to add me as a collaborator and then have me create a local branch and submit PRs to the remote to be approved by you or another member of the team. Unless you'd rather test me first, which is completely fair.

[collinhundley]

@KacheFlowe If you have new changes to add now, go ahead and submit a PR and I'll take a look at it. It's been a busy few weeks for me but I'm getting back into a normal routine so I'll have more time to dedicate to Swift AI again. If you have lots of contributions that you'd like to make, I'm not opposed to adding more collaborators if that's easier.