Dynamics, Modeling, and Machine Learning
This post intuitively introduces concepts from physics that are helpful when thinking about modeling systems and what machine learning fundamentally aims to accomplish. I learned of these ideas in various forms while studying machine learning and analytical mechanics, but I have not seen a resource where these things are stated simply and directly. There are many papers about these topics, but few are accessible to laypeople. This is my attempt to make the intuition behind these powerful ideas clear. I generally stick to having the simplest bits and visualizations at the beginning of each section and delve into the mathematical details further in. If you are looking for the lightest possible treatment of these ideas, just read a section until the details become opaque to you and move on to the next section.
The main insights are ( don’t worry if you don’t know the definitions yet ):
- A sample is a projection.
- Projections are entropic.
- The entropy of projections can be reduced by multiple samples.
- The effectiveness of sampling is related to the sample diversity.
- The diversity of samples is related to movement in higher dimensions.
- Complex dynamics in lower dimensions can be described by simpler dynamics in higher dimensions.
These insights do not correspond to a particular section. Different sections will explore various ways in which these insights interact to account for various aspects of modeling the world with mathematics.
Table of Contents
- Prelude
- Flipping a Coin
- Projection and Movement
Prelude
Flipping a Coin
Projection and Movement
Consider a cone floating in a pond.