Spaced Repetition Over Variational Autoencodings
How much do we actually retain when we read stuff, listen to podcasts, take some class? Sometimes it frustratingly feels like it all just passes through. How much do I actually retain? As Paul Valéry’s weird character, Monsieur Teste, says, looking back on his life: “I can now recall a few hundred faces, two or three great spectacles, and the substance of perhaps twenty books. I have not retained the best nor the worst of these things: what could stay with me did.” Can we be less passive with what sticks?
Machine learning has this concept of variational autoencoders. Those are neural network architectures that efficiently encode information (in some structure with much less bits) and then decode it with more or less fidelity to the original input. That “space” in which the information is efficiently stored by the encoder and retrieved by the decoder is called the latent space. The variational part just means that the decoder can sample from that latent space to obtain variations around the same kind of input the encoder has encoded. It therefore can afford to be somewhat generative, creating new from old. It chews up stuff, digests it, and then regurgitates it with its own mark on it. I bring up VAEs because I see them as sort of a metaphor for the kind of more purposeful retaining I want to do. By training a decoder alongside an encoder, we end up not only with a better latent representation of knowledge, but also more generative ability around it.
Hence the importance of writing about and around what you learn and experience, hence blogging. It makes things stick better. The next step, once you’ve chewed on things once, is to go back over them, using the spaced repetition method, whereby you create flashcards around some topic you wish to remember, and you exponentially space out the times at which to go through your flashcards. There’s a good amount of evidence for this method, although no one really knows why. Perhaps neuronal connections form best when stimulated at a decaying rate? In any case, we need software that creates flashcards from blog posts, and pings them to you at expanding intervals to cement your knowledge. One could even make a business out of it by embedding a social network in such software: follow people’s posts, subscribe to their flashcards. If someone’s good at distilling knowledge, and good at making flashcards, why couldn’t they charge for it? Turn learning into an activity with positive externalities; feed your nuggets of knowledge right into the interest graph.
