Robust learning rate finder with Kalman smoothing
Kalman smoothing can be applied to the learning rate range test to produce smooth learning rate curves from which a learning rate can be chosen.
Spike distance function as a learning objective for spike prediction
A learning objective that embraces the nature of spikes (or events times, more generally): discrete points in continuous time. This learning objective allows models to predict the timing of spikes with both high and low temporal precision. For comparison, predictions when using the Poisson learning objective are also shown above. [paper page] [paper pdf]
Neural Networks and Color
This is a project to see if anything can be learnt by drawing parallels between human color perception and neural networks trained for vision tasks.
Motivating ELBO From Importance Sampling
The evidence lower bound expression appears naturally when you try to sample the posterior distribution with an approximate distribution. I think this way of arriving at the evidence lower bound is more intuitive and reveals more about why concessions are being made.
Inside Neural Network Training
Some videos showing how weights, activations and gradients change at every step as a network is trained.
Anki Notes
I've exported some of my Anki decks. Most of the content is on the answer side of each card. If you aren't using Anki, I invite you to read Michael Nielsen's argument for it.
2021's Summer of Math Exposition
The following are some posts made for 3Blue1Brown's Summer of Math Exposition. They are attempts to make clear and memorable math explainers for a wide audience.