Improve discussion of derivatives #266

Open
sandwichmaker opened this Issue Mar 26, 2017 · 0 comments

Comments

Projects
None yet
1 participant
Contributor

sandwichmaker commented Mar 26, 2017 edited

List of things to add:

  • Why does the quality of derivatives matter?
  • Discuss, forward v/s backward automatic differentiation and relation to backprop, impact of large parameter block sizes on differentiation performance.
  • Inverse function theorem
  • Calling iterative routines.
  • Reference to how numeric derivatives lead to slower convergence.
  • Pitfalls of Numeric differentiation.
  • Ill conditioning of numeric differentiation/dependence on curvature.

sandwichmaker self-assigned this Mar 26, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment