Transition Matrices
What is a transition matrix?
- A transition matrix T shows the transition probabilities between the current state and the next state
- The columns represent the current states
- The rows represent the next states
- The element of T in the ith row and jth column gives the transition probability tij of :
- the next state being the state corresponding to row i
- given that the current state is the state corresponding to column j
- The probabilities in each column must add up to 1
- The transition matrix depends on how you assign the states to the columns
- Each transition matrix for a Markov chain will contain the same elements
- The rows and columns may be in different orders though
- E.g. Sunny (S) & Cloudy (C) could be in the order S then C or C then S
What is an initial state probability matrix?
- An initial state probability matrix s0 is a column vector which contains the probabilities of each state being chosen as the initial state
- If you know which state was chosen as the initial state then that entry will be 1 and the others will all be zero
- You can find the state probability matrix s1 which contains the probabilities of each state being chosen after one interval of time
- s1 = Ts0
How do I find expected values after one interval of time?
- Suppose the Markov change represents a population moving between states
- Examples include:
- People in a town switching gyms each year
- Children choosing a type of sandwich for their lunch each day
- Suppose the total population is fixed and equals N
- You can multiply the state probability matrix s1 by N to find the expected number of members of the population at each state
Examiner Tip
- If you are asked to find a transition matrix, check that all the probabilities within a column add up to 1
- Drawing a transition state diagram can help you to visualise the problem
Worked example
Each year Jamie donates to one of three charities: A, B or C. At the start of each year, the probabilities of Jamie continuing donate to the same charity or changing charities are represented by the following transition state diagram: