there are three main categories of Design Patterns as decreed by the ‘Gang of Four’1 (the authors of a seminal work Design Patterns | Elements of Reusable Object-Oriented Sofware.
I remember when using Emacs itself was a huge struggle for me. But now I have just sudo apt install emacs'd this vanilla install and I am already off to the races.
Anyways, I'll probably slim down this prose at a later date when I find it cringe and too verbose; but for now I am having a terrific time thwacking away at a Drunkdeer A75 Pro (thanks Aarav).
I've opted to scribble here as opposed to in a README this time.
Honestly, the diagrams that I wish to reproduce already exist here. Currently this page is in construction and probably will be until I finish my Doctorate.
“Memory is the mother of all wisdom."
— Aeschylus
Babbage's Big Brain
Memory as a Hierarchy — Not a Monolith
Hierarchy exists for two intertwined reasons:
Physics – Smaller structures are faster and nearer to ALUs but hold less data; larger structures store more but are farther away and thus slower.
Economics – Fast memory costs disproportionately more per byte.
An efficient system arranges multiple layers so that
> the majority of accesses hit the small, fast part,
> while the bulk of bytes reside in the large, cheap part.
Recall that a Neural Network follows the following construction:
Pass data (forward) through model to get predicted values
Calculate loss with predicted values against labels
Perform backpropagation w.r.t each weight / bias to get the direction in which to move that weight such that it moves closer to the global minima
Update parameters with gradients using an optimiser.
momentum
ball's pace slows down
this makes total fkn sense! if the gradient signs are the same, increasing your confidence in that direction and move further.
you want to take less steps over all
