Stochastic Series - Pt 5 - Introducing q
By now we know that a stochastic process must have a specific state at time n. We also know that our process has a transition matrix (P) that represents the "rules" for going from one state to another.
If you read Pt. 3, you also have an idea of how to do transitions like:
"What is the probability of Day 1 being Windy IF Day 0 is Sunny?"
The above problem is quite simple: we only need to look at the transition matrix to figure out what the probability is. In other words, there is no need to do any multiplication.
Assume our states are ordered as:
Sunny, Windy, Rainy
P =
Sunny Windy Rainy
Sunny 0.5 0.3 0.2
Windy 0.2 0.5 0.3
Rainy 0.1 0.4 0.5
Answer: 0.3 or 30%
Actually, that is not the whole truth because we are using a predefined row vector that looks like this:
q0 = (1, 0, 0)
The vector above reflects the probability distribution for each state on Day 0: Sunny, Windy, and Rainy, where Sunny has probability 1 (100%).
We use q0 together with our transition matrix to calculate the probability distribution for Day 1:
q1 = q0 × P
If we only care about the probability of transitioning from Sunny to Windy, we calculate:
(1 × 0.3) + (0 × 0.5) + (0 × 0.4) = 0.3
As you can see, we get the same answer as before, so why do we even care about q?
Well, it turns out that if we are not sure what state we are transitioning from, it becomes quite handy.
Let's say we don't know what state we are transitioning from, so we assume that all three states are equally likely:
q0 = (1/3, 1/3, 1/3)
Now let's ask:
What is the probability of transitioning into a Windy state?
(1/3 × 0.3) + (1/3 × 0.5) + (1/3 × 0.4) = 0.4
So the probability of transitioning into a Windy state is 40%.
But q0 × P actually calculates the probability for all states, not just Windy.
For Sunny:
(1/3 × 0.5) + (1/3 × 0.2) + (1/3 × 0.1) = 0.2667
For Windy:
(1/3 × 0.3) + (1/3 × 0.5) + (1/3 × 0.4) = 0.4
For Rainy:
(1/3 × 0.2) + (1/3 × 0.3) + (1/3 × 0.5) = 0.3333
We put those three probabilities together and get our new row vector:
q1 = (0.2667, 0.4, 0.3333)
So q1 represents the probability distribution for Day 1.
We can then use q1 to calculate the probability distribution for Day 2:
q2 = q1 × P
And continue in the same way:
q0 --P--> q1 --P--> q2 --P--> q3
Each q represents the probability distribution over all possible states at that point in time, while P contains the rules for transitioning from one state to the next.