Stocastic Series - Pt 3 - Multi-Step Transition Probabilities

21 Sep, 2026

Introduction

One of those things that is easy to understand conceptually but hard to grasp once we look at the math is doing multi-step transition probabilities.

Most articles out there do not fully paint the picture of what is actually going on under the hood. So in this guide I am going to try my best to help you understand the WHY of things more than the HOW.

Predicting the weather

The concept of a day

For this example, we treat these as three separate categories, so each day has exactly ONE weather category.

We start each transition chain with Day 0, so to get from Day 0 to Day 1 we think of the chain like this:

Day 0 -> Day 1

The above is read as: Starting at Day 0 and TRANSITIONING into Day 1. Also note that there are essentially two days, Day 0 and Day 1, but only one transition.

Day 0 -> Day 1 -> Day 2

The above is read as: Starting at Day 0 and TRANSITIONING into Day 1 and then into Day 2. As you can see, there are 3 days but only two transitions.

The concept of a state

A state in a stochastic process is a specific value or condition that the random system can occupy at any given point in time. Since we are working with the weather, we have three different states:

1 = Sunny

2 = Windy

3 = Rainy

Each state is associated with its own unique number. If we say that tomorrow will have state 2, we are saying that tomorrow will be windy.

We now expand our way of expressing transitions between days and their weather:

Day 0 (Sunny) -> Day 1 (Rainy)

Read as:

Starting at Sunny Day 0 and TRANSITIONING into Rainy Day 1.

Transition matrix P

We need a way to describe the probability of transitioning from one type of weather into another so we can answer the question:

"What is the probability of Day 1 being windy IF Day 0 is Sunny?"

We can express the probabilities using something called a transition matrix.

A transition matrix is a collection of transition probabilities that we can read as a table.

First, let's talk about how to read a matrix in general, which is basically a collection of rows and columns.

Read it like this:

What's the value of Row 2, Column 3?

First you navigate to the second row (0.2, 0.5, 0.3), and then you go to the third column and you will find the value 0.3.

Each value in the transition matrix represents a probability. Let's add row and column names to it.

Read it like this:

What's the probability of rain tomorrow (state 3), given it's sunny today (state 1)?

When transitioning from one state to another, we read the ROWS in the matrix as the FROM state and the COLUMNS as the TO state.

Since we are transitioning FROM sunny (state 1) TO rainy (state 3), we first navigate to ROW 1 and then COLUMN 3. There we find the value 0.2.

Another way to express this is:

Read it as:

Probability of going from sunny (state 1) to rainy (state 3) is 0.2.

Days, States and Transition Matrices

Each day has a state, and going from one state into another is called a transition. We use a transition matrix to determine what the probabilities are of going from one specific state to another.

Now comes the hardest fundamental part of understanding states, days and transition matrices. We want to learn this in the right order, so we are going to look at Day 0.

Day 0

Day 0 is a special day because it's the day we start with and we know what state we are starting from. Therefore, the zero-step transition matrix looks a bit different:

If we start with Sunny and take zero steps, there is a 100% probability of Sunny and a 0% probability of Windy or Rainy.

It represents no transition having happened yet. It does not mean the weather cannot change once we take a step.

Day 1

Day 1 is different because now we need a transition matrix. It needs to be given to us in some way. But for now, we are going to use this one:

It's the same one you saw before, and I refer to this as the base transition matrix. For this example, we assume that the transition probabilities stay the same, so we keep using it for all days going forward from Day 1.

Many transitions

The previous example only used 1 step forward into the future, but what about multiple steps? Let's say we know the weather on Day 0 and we want to know the probability of Rainy on Day 2.

Same one-step transition matrix:

So how should we think about this problem?

Well, when we only did one step going from Sunny to Rainy, we only needed to look up one entry:

Day 0 (Sunny) → Day 1 (Rainy)

But now we need to add another day:

Day 0 (Sunny) → Day 1 (Rainy) → Day 2 (Rainy)

But wait, is that really correct? The above transition flow can be interpreted as:

If Day 0 is Sunny, what is the probability that both Day 1 and Day 2 will be Rainy?

And that's not what we want. We actually want:

Day 0 (Sunny) → Day 1 (any weather) → Day 2 (Rainy)

We don't want just the probability of going from Sunny → Rainy → Rainy. That is only one of the possible paths.

So we need to account for all of the possible weather on Day 1.

Matrix multiplication

FROM Day 0 TO Day 1

To answer the question, we need to use matrix multiplication.

Day 0 (Sunny) → Day 1 (any weather) → Day 2 (Rainy)

We know that Day 0 is Sunny, and transitioning into any weather means we need to calculate the probability for each weather coming from Sunny on Day 1:

Day 0 (Sunny) → Day 1 (Sunny)

Day 0 (Sunny) → Day 1 (Windy)

Day 0 (Sunny) → Day 1 (Rainy)

Remember, the ROWS represent the FROM state and COLUMNS represent the TO state.

For Day 1, these probabilities are simply ROW 1: [0.5, 0.3, 0.2]

So the probabilities become:

Day 0 (Sunny) → Day 1 (Sunny) = 0.5

Day 0 (Sunny) → Day 1 (Windy) = 0.3

Day 0 (Sunny) → Day 1 (Rainy) = 0.2

FROM Day 1 TO Day 2

Now we want to make another transition from Day 1 to Day 2.

Our row vector for Day 1 is:

To calculate the probabilities for Day 2, we multiply this row vector with COLUMN 1, 2 and 3 of our base transition matrix.

Any (Day 1) → Sunny (Day 2, Col 1):

Any (Day 1) → Windy (Day 2, Col 2):

Any (Day 1) → Rainy (Day 2, Col 3):

So our new row vector becomes:

Now let's look at the problem we are trying to solve again:

Day 0 (Sunny) → Day 1 (any weather) → Day 2 (Rainy)

Since we only care about the probability of ending in rain, we only need the third value in our row vector:

So the answer to the question:

Day 0 (Sunny) → Day 1 (any weather) → Day 2 (Rainy)

is 0.29.

Or rather:

Read it like:

Probability of going from Sunny to Rainy in 2 transitions is 29%.

FROM Day 2 TO Day 3

What if we wanted to go one step further and calculate the probabilities for Day 3?

We already calculated the row vector for Day 2:

To get Day 3, we simply do the same thing again. We use our Day 2 row vector and multiply it with COLUMN 1, 2 and 3 of our base transition matrix.

Any (Day 2) → Sunny (Day 3, Col 1)

Any (Day 2) → Windy (Day 3, Col 2)

Any (Day 2) → Rainy (Day 3, Col 3)

So our new row vector becomes:

And if we only care about the probability of going from Sunny on Day 0 to Rainy on Day 3:

Read it like:

Probability of going from Sunny to Rainy in 3 transitions is 32.5%.

And this pattern keeps going. Every time we multiply our current row vector with the base transition matrix, we move forward one more day, or one more transition.