In algorithmic information theory (AIT), Levin's coding theorem (which should be much more widely taught in Physics!) predicts that, upon uniform random sampling of programmes, a universal Turing machine (UTM) will be exponentially biased towards outputs with low Kolmogorov complexity. In this talk I will provide evidence for a similar exponential bias towards descriptional simplicity (low Kolmogorov complexity) in biological evolution. A similar Occam's razor-like bias helps explain why deep neural networks generalize well in the overparameterised regime, where classical learning theory predicts they should badly overfit. I will discuss how these principles from AIT fit into a wider discussion about the use (and abuse) of Occam's razor in science.
Philosophy of Physics Seminar Convenor for TT22: Simon Saunders | Philosophy of Physics Group Website