thoughts on better neural net

[andrewljohnson]

The current neural net uses something really made for handwritten digit classification (MNIST).

We could do stuff like use Alexnet instead, or implement the neural nets described in Mnih/Hinton too. Literature also describes using a sequence of pre and post processing neural nets, where you can fill in gaps in road networks.

Expanding on these vague comments, there is a whole body of literature about how to use CNNs, RNNs, global topology, lidar elevation data, and much more to improve the accuracy of satellite imagery label. We should be able to get above 90% on the pixel level, just using semi-local RGB data, and push past that with multiple neural nets, more data, and other documented improvements in the last 2-3 years since Mnih's thesis.

See the README for a list of readings.

[andrewljohnson]

@lacker says to make a simpler neural net that works better with the given amount of IR training data, so the analysis overfits less.

Then expand the layers/data.

Don't just use some complicated neural net unsuited to the task.

[lacker]

Yeah, I think you want to get to a state where "it works, just could be better" before scaling up. That way you can figure out how long you need to train, and keep params in a state where you don't overfit. I would try something real simple, like one convolution layer plus one softmax layer, and see if you can make it "work". By "work" I mean significantly better than random, like 80% accuracy, and not much of an overfitting problem.

[andrewljohnson]

I think we're now in a state of "works with the simplest network." A single relu or single convolutional relu with a maxpool.