18-Learning-From-Examples

[redblobgames]

Let's discuss visualizations and code for chapter 18 here. What are the main ideas from the chapter that could be turned into interactive/animated visualizations?

[redblobgames]

We usually think about interactive visualizations solving the problem for you. The reader provides the inputs, and the page runs the algorithm and provides the output. Also consider whether there are some concepts where you want to invert this, and have a fixed input problem, and ask the reader to "run" some part of the algorithm by clicking on the expected output.

[Ghost---Shadow]

18.5 Decision-Tree-Learning DecisionTreeLearner learning.py
18.8 Cross-Validation cross_validation learning.py
18.11 Decision-List-Learning DecisionListLearner learning.py
18.24 Back-Prop-Learning BackPropagationLearner learning.py
18.34 AdaBoost AdaBoost learning.py

Here are some of my favorites

1. http://playground.tensorflow.org/
2. http://neuralnetworksanddeeplearning.com/chap4.html (Near the end of the page)
3. http://colah.github.io/posts/2014-10-Visualizing-MNIST/
4. http://distill.pub/2016/handwriting/

[preethi26]

@redblobgames, sir, I have just started working on chapter 18: learning from examples.
The plan I have thought of is as follows:

1. Implement the decision tree learning, decision list learning, backpropagation learning.
   Visualisations:
   Decision tree learning: Building a tree with the user's help. The input is fixed, and at each step, the user will be asked to choose from the set of attributes, the best attribute. As we know the solution, we can correct the user by showing why his/her answer was not the best by providing our solution.
   Decision list learning: similar to the above description, (letting the user try and correcting if he/she is wrong ).
   Back propagation learning: The goal will be to show the neural network with best possible weights in the form of a graph (like here. mentioned by @Ghost---Shadow in the previous comment. I was thinking of the same. )We can have fixed input on a simple example of 'n'-classes. The stopping criteria and the training set can be taken from the user as input. We can show after each epoch how the training set examples are being classified and the weights being changed.

2. implementing Support vector machines:
   visualisation: the user can give input in the form of making few points belonging to different classes.
   we can show the step by step procedure, how we get the solution in the visualisation.

3. There are few concepts that I think will be helpful if explained.( did not think of visualisation yet)
   a. differentiating univariate and multivariate regression. explaining the need of multivariate.
   b. concepts in the non-parametric models, ex: the nearest neighbour model.

4. There are two other algorithms described in the book, Cross-Validation-Wrapper and the Ada-Boost.
   I am not very sure how to proceed with the visualisation part related to these two algorithms.

I will be glad if I can get some suggestions and reviews so that we can proceed without wasting the effort that we put in :)

[redblobgames]

@preethi26 I think that's a great set of visualizations to try! Asking the user to guess the answer is a technique I don't see very often. I first saw it here (requires Flash) — it gives you a function, and the user draws the derivative of the function, and then it shows you how closer you got. It really made me think about derivatives much more than if it showed me the answer right away! There is another example here where you have to guess the answer and then it shows it to you. I've also seen it used for non-technical topics here.

Many ideas we won't have right away but after working on one of the visualizations you'll have ideas for more. :)

There may be topics that don't work well with interactive visualizations, and that's ok.

@Ghost---Shadow love those links! Colah has lots of nice articles on the main page. ConvNet has an interactive demo. This page is mostly explanation but also has a few interactive parts.

[Rishav159]

I love the Decision tree learning idea from @preethi26 This seems like a great way to demonstrate Decision tree learning. The reader will be forced to think about the best parameter that divides the dataset. We can show proper feedback after that. After the decision tree is learned, we can allow the user to give enter a new row of data and watch the tree predict it.
@preethi26 If you have made some progress in this diagram, kindly let me know so we can collaborate.

[preethi26]

Sure @Rishav159
I have created a branch and worked on it separately,
here is the link, decision-tree. I would like to collaborate with you.
Have a look at it :)

[Rishav159]

Sure

[Rishav159]

Can you please explain what the diagram does ?

[preethi26]

Sure @Rishav159,

Open the console ( ctrl + shift + i )
In the diagram: Squares represent the examples and the circles represent the classes.
reload the web page.

You will find the constructed decision tree's nodal forks ( where each fork is a class ) in the descending order. For example, if it is: (1,10,6,5) in the descending order, it means that the tree has the following integers as the nodal branches where the top most node is '5' and then comes '6', '10', '1' in level order traversal.

click on the blue circle which starts the program.
Now you can see the tree in the form of forks as the last output in the console. Say it is 7 10 9 5.

click on 5 from the green circles. the tree expands. Remember, after every level of the tree expansion you need to click on which node you want to expand which updates the classes circles and then click on what that could be.

After you click 5, the tree expands, choose the unexpanded node ( which causes the circle 5 to disappear. )then click 9 from the green coloured class list.

Now the tree could have more than one expandable node, choose one by clicking it (both 5 and 9 disappear ) and then click 10/7 ( any one could be right ), the tree expands again. now click the other node that you want to expand ( the correct one disappears here it is 10 )and click 7.

Now it's done.
I am sorry that the code is in debugging state, I hope you are clear now. If you don't get anything please do tell me :)