By 苏剑林 | October 06, 2017
The master equation is an important method for modeling stochastic processes. It represents the differential form of a Markov process. In our field, it is one of our primary tools; to put it more broadly, even quantum mechanics and statistical mechanics are essentially special cases of the master equation.
However, after reading several textbooks—such as A Modern Course in Statistical Physics and my supervisor's Stochastic Dynamics of Biological Systems—I found that their derivations of the master equation are quite vague. They focus heavily on explaining the meaning of the results but do not clarify the conceptual origins of those results, making the process difficult to find convincing. This is further evidenced by questions on Zhihu, such as "How to understand the derivation process of the master equation for Markov processes?", which have yet to receive satisfactory answers.
Markov Process
The master equation is used to describe Markov processes. A Markov process can be understood as the property of "memorylessness" in motion. In simple terms, the probability distribution at the next moment depends only on the current state and is independent of its history. Expressed in probability formulas (considering only continuous-type probabilities, where $p$ is the probability density):
\begin{equation}\label{eq:maerkefu}p(x,\tau)=\int p(x,\tau|y,t) p(y,t) dy\end{equation}
Here, the integration is over the entire space. $p(x,\tau|y,t)$ is called the transition probability, which represents the probability density of reaching position $x$ at time $\tau$, given that the system was at position $y$ at time $t$. The physical meaning of this equation is quite intuitive and requires no further explanation.
Although equation \eqref{eq:maerkefu} is intuitive, using it for modeling presents two significant difficulties:
1. To model, one must write down $p(x,\tau|y,t)$, but in reality, it is very difficult to directly formulate a reasonable transition probability.
2. Even if $p(x,\tau|y,t)$ is written down, the equation is an integral equation, and our understanding of integral equations is much less developed than that of differential equations.
Therefore, it is necessary to derive its differential equation form.
The Master Equation
Let $\tau = t + \epsilon$:
\begin{equation}\label{eq:maerkefu-2}p(x,t+\epsilon)=\int p(x,t+\epsilon|y,t) p(y,t) dy\end{equation}
Taking the limit $\epsilon \to 0$ and keeping terms up to the first order of $\epsilon$, we have:
\begin{equation}\label{eq:yueqian}p(x,t+\epsilon|y,t)=\delta(x-y)+\epsilon \tilde{W}(x,y,t)\end{equation}
where we use the fact that $p(x,t|y,t)=\delta(x-y)$, and
\begin{equation}\tilde{W}(x,y,t)=\left.\frac{\partial p(x,\tau|y,t)}{\partial \tau}\right|_{\tau=t}\end{equation}
Expanding both sides of equation \eqref{eq:maerkefu-2} to the first order of $\epsilon$, we get:
\begin{equation}p(x,t)+\epsilon\frac{\partial p(x,t)}{\partial t}=\int \left[\delta(x-y)+\epsilon \tilde{W}(x,y,t)\right] p(y,t) dy\end{equation}
Thus,
\begin{equation}\label{eq:zhufangcheng-1}\frac{\partial p(x,t)}{\partial t}=\int \tilde{W}(x,y,t) p(y,t) dy\end{equation}
Facilitating Modeling
Readers might notice that equation \eqref{eq:zhufangcheng-1} is not the usual form of the master equation we see in textbooks. This is because it is not convenient for modeling. Let us revisit equation \eqref{eq:yueqian}. Note that we have:
\begin{equation}\int p(x,t+\epsilon|y,t) dx = 1\end{equation}
Therefore, combined with equation \eqref{eq:yueqian}, it must be that:
\begin{equation}\label{eq:yueshu}\int \tilde{W}(x,y,t)dx = 0\end{equation}
The modeling process in scientific research works in reverse: one needs to write down the form of the master equation first and then solve it. That is to say, in order to use equation \eqref{eq:zhufangcheng-1} for modeling, one needs to write out $\tilde{W}(x,y,t)$, which must satisfy the constraint of equation \eqref{eq:yueshu}. However, it is very difficult to construct a $\tilde{W}(x,y,t)$ out of thin air that simultaneously satisfies this constraint.
Fortunately, we can use a trick to remove this constraint. First, we write an arbitrary function $W(x,y,t)$, then consider:
\begin{equation}\begin{aligned}&\int W(x,y,t)dx\\
=&\iint \delta(y-z) W(x,z,t)dxdz \quad (\text{Next, swap } x \text{ and } z)\\
=&\iint \delta(y-x) W(z,x,t)dzdx\end{aligned}\end{equation}
Thus, we can let
\begin{equation}\tilde{W}(x,y,t) = W(x,y,t) - \int \delta(y-x) W(z,x,t)dz\end{equation}
Then it automatically satisfies equation \eqref{eq:yueshu}. The final form of the master equation becomes:
\begin{equation}\label{eq:zhufangcheng-2}\frac{\partial p(x,t)}{\partial t}=\int \Big[W(x,y,t)p(y,t)-W(y,x,t)p(x,t)\Big] dy\end{equation}
This is the form of the master equation commonly seen in textbooks. In this form, there are no special constraints on $W(x,y,t)$, making it easy to use for modeling. As for the interpretation of the physical meaning of $W(y,x,t)$, that is a separate discussion. A similar derivation can be applied to the discrete version of the master equation, so it will not be repeated here.
A Brief Evaluation
From the above process, we can see that the master equation appears in the form we usually encounter because that specific form is more convenient for modeling. As for the interpretations of the results, strictly speaking, they are often imposed as an afterthought rather than being part of the derivation process itself. Many works focus on the physical interpretation of the result while neglecting the explanation of the equation's formal origin, which is one of the main reasons for the difficulty in understanding it. I hope these words serve as a helpful supplement.