Markov Chains

My project was to research and write a report on Markov chains – a category of random processes. While a random process is simply a probability distribution over some states which changes over time, a Markov chain adds the condition that the transition probabilities between states are fixed.

We notice when observing Markov chains that many chains tend towards some fixed distribution – the “stationary distribution”. This distribution has the property that it does not change over time. My report considers the conditions under which such a distribution exists, the conditions under which a chain converges to this distribution, and how quickly chains converge.

Finally, I applied the theory developed in the paper to the board game Monopoly. By considering each square as a state, and calculating (approximate) fixed transition probabilities between each square, I was able to find the stationary distribution. Since Monopoly satisfies the conditions for convergence, this gives the long-term probability of landing on any given square. This can be used for determining which properties are the most valuable.