Regularisation

Feedforward Deep Neural Networks

2025-07-21 by [Michael Neilsen Aayush Bajaj]

This page includes ⊕ my Chapter notes for the book by Michael Nielsen.

chapter 5: why are deep neural networks hard to train?

2025-04-15

vanishing‑exploding gradients relu dropout regularisation overfitting augmented‑data

given the findings of the previous chapter (universality), why would we concern ourselves with learning deep neural nets?
- especially given that we are guaranteed to be able to approximate any function with just a single layer of hidden neurons?

well, just because something is possible, it doesn't mean it's a good idea!

considering that we are using computers, it's usually a good idea to break the problem down into smaller sub-problems, solve those, and then come back to solve the main problem.

chapter 6: deep learning

2025-04-14 (updated: 2025-07-25)

cnn theano relu dropout regularisation overfitting augmented‑data

Regularised Regression

2025-02-13

bayesian map posterior regularisation overfitting bias‑variance

This page is for the closed form solutions (where they exist) and approximation solutions to Regularised Regressions.

We will also understand that regularisation is sensible artifact once we consider its MAP (maximum a posteriori) derivation.