单击此处编辑母版标题样式,单击此处编辑母版文本样式a,第二级,第三级,第四级,第五级,*,*,排 队 论,计算机科学与技术学院,2006年9月,1,第二章 马尔可夫链Markov Chain,一 离散时间的马尔可夫链,二 连续时间的马尔可夫链,三 生灭过程,四 泊松过程,2,随机过程,设R表示全体实数，对任一实数t，X(t)是一个随机变量，那么随机变量族X(t),tR称为一个随机过程,随机过程分类的三个主要因素,状态空间,离散状态空间称之为链Chain,连续状态空间,时间t,离散时间常把随机变量记做Xn，称之为随机序列stochastic sequence,连续时间X(t),称为随机过程 stochastic process),不同时间上随机变量之间的依赖关系,3,马尔可夫链的产生,1907年马尔可夫发表论文，定义并研究了一种随机过程的性质，就是我们现在所称的马尔可夫过程,离散状态空间的马尔可夫过程就是马尔可夫链,KleinrockWhat he created was a simple and highly useful form of dependency among the random variables forming a stochastic process,which we now describe,4,第二章 马尔可夫链第一节 离散时间的马尔可夫链,5,1 离散时间的马尔可夫链,定义,设X=Xn，n=0,1,2,3.是一个随机过程，Xn取值为0,1,2,3，即状态空间E=0,1,2,3，,用“Xn=i表示时刻n系统X处于状态i这一事件。,称 pij(n)=P(Xn+1=j|Xn=i)为在事件“Xn=i出现的条件下，事件“Xn+1=j出现的条件概率。,6,1 离散时间的马尔可夫链,定义,如果对任意的非负整数i1,i2,.,in-1,i,j及一切n0有,P(Xn+1=j|Xn=i,Xk=ik,k=0,1,2,n-1),=P(Xn+1=j|Xn=i),=pij(n),那么称X是一离散时间的马尔可夫链(Markov Chain)。,7,(Kleinrock)The Markov property insists that the past history be completely summarized in the specification of the current state.,Markov,8,马尔可夫性举例,例1,统计通过路口的车辆。设Xn表示时刻n时已通过的车辆数，n=10的车辆数，要想知道n=15时的车辆数，只要统计10-15时之间通过的车辆数情况就可以了。因此，车流具有马尔可夫性。,Xn,n,0,10,15,500,?,9,马尔可夫性举例,例2,假定每个细菌经过一个时间单位分裂一次或走向死亡，如果时刻i时刻的细菌数Xi，求时刻jji时的细菌数Xj，只要知道从时刻i时的情况推算就可以了，因而Xn也是一个马尔可夫链,0,m,n,Xn,时刻,X,i,X,j,10,2 齐次马尔可夫链,当pij(n)与起始时刻无关时，那么称为齐次的马尔可夫链。此时可将pij(n)记为pij,称pij为系统的一步转移概率。,把一布转移概率写成矩阵形式：,称为一步转移矩阵,矩阵元素性质：,11,3n步转移概率,齐次马尔可夫链中，称,p,ij,(n),=P(X,n,=j|X,0,=i)=P(X,m+n,=j|X,m,=i),为马尔可夫链X的,n步转移概率,也可写成矩阵的形式,称为,n步转移矩阵,12,4 K-C方程,i,j,*,*,*,*,.,n,m,时间,X,n,Kolmogorov,Chapman,13,5 初始分布 绝对概率,假设记piP(X0=i)，那么有,称pi，iE为齐次马氏链X的初始分布。,P(Xn=i)记做 ,为齐次马尔可夫链的绝对概率,14,5 初始分布 绝对概率,第n步绝对概率初始分布n步转移概率,15,6 平稳分布,假设极限 存在，且,那么称为i系统的平稳分布,X,0,初始分布p,i,0,k,X,k,瞬时分布,平稳分布,i,16,6 平稳分布,书 第27页,17,6 平稳分布,这就是离散时间的马尔可夫链求平稳分布的公式,18,平稳分布举例,齐次马氏链一步转移概率矩阵为,0,2,1,3/4,1/4,1/4,3/4,1/4,1/4,1/2,19,平稳分布举例,根据求平稳分布公式,得,20,平稳分布举例,解之得：,为平稳分布的值,0,2,1,3/4,1/4,1/4,3/4,1/4,1/4,1/2,21,平稳分布举例,三种初始分布对瞬时概率和平稳分布的影响,A.初始分布1,0,0:,B.初始分布0,1,0:,n,0,1,2,3,4,1,0,0.250,0.187,0.203,0.20,0,0.75,0.062,0.359,0.254,0.28,0,0.25,0.688,0.454,0.543,0.52,n,0,1,2,3,4,0,0.25,0.187,0.203,0.199,0.20,1,0,0.375,0.250,0.289,0.28,0,0.75,0.438,0.547,0.512,0.52,22,平稳分布举例,C.初始分布0,0,1:,可以看出,不管初始分布如何，系统的瞬时概率总是很快的趋近于平稳分布的取值,瞬时分布趋近平稳分布的速度与转移矩阵P有关,n,0,1,2,3,4,0,0.25,0.187,0.203,0.199,0.20,0,0.25,0.313,0.266,0.285,0.28,1,0.50,0.500,0.531,0.516,0.52,23,+x*u$qZnWkShPeMaJ7F4C1z)w&t!pYmUjRgOcL9I6E3B+y(v%r#oXlTiQeNbK8G5D2A-x*u$qZnVkShPdMaJ7F4C0z)w&s!pYmUjRfOcL9H6E3B+y(u%r#oWlTiQeNbJ8G5D1A-x*t$qYnVkSgPdMaI7F3C0z)v&s!pXmUiRfOcK9H6E2B+y(u%rZoWlThQeNbJ8G4D1A-w*t$qYnVjSgPdLaI7F3C0y)v&s#pXmUiRfNcK9H5E2B+x(u$rZoWkThQeMbJ7G4D1z-w*t!qYnVjSgOdLaI6F3C0y)v%s#pXlUiRfNcK8H5E2A+x(u$rZnWkThPeMbJ7G4C1z-w&t!qYmVjRgOdL9I6F3B0y(v%s#oXlUiQfNbK8H5D2A+x*u$rZnWkShPeMaJ7G4C1z)w&t!pYmVjRgK8H5E2A+x(u$rZnWkThPeMbJ7G4C1z-w&t!qYmVjRgOdL9I6F3B0y)v%s#oXlUiQfNcK8H5D2A+x*u$rZnWkShPeMaJ7G4C1z)w&t!pYmVjRgOcL9I6E3B0y(v%r#oXlTiQfNbK8G5D2A-x*u$qZnVkShPdMaJ7F4C1z)w&s!pYmUjRgOcL9H6E3B+y(v%r#oWlTiQeNbK8G5D1A-x*t$qZnVkSgPdMaI7F4C0z)v&s!pXmUjRfOcK9H6E2B+y(u%r#oWlThQeNbJ8G5D1A-w*t$qYnVkSgPdLaI7F3C0z)v&s#pXmUiRfOcK9H5E2B+x(u%rZoWkThQeMbJ8G4D1z-w*t!qYnVjSgPdLaI6F3C0y)v&s#pXlUiRfNcK9H5E2A+x(u$rZoWkThPeMbJ7G4D1z-w&t!qYmVjSgOdL9I6F3B0y)v%s#oXlUiQfNcK8H5D2A+x*u$rZnWkThPeMaJ7G4C1z-w&t!pYmVjRgOdL9I6E3B0y(v%s#oXlTiQfNbK8H5D2A-x*u$qZnWkShPdMaJ7F4C1z)w&s!pYmUjRgOcL9I6E3B+y(v%r#oXlTiQeNbK8G5D2A-x*t$qZnVkShPdMaI7F4C0z)w&s!pXmUjRfOcL9H6E2B+y(u%r#oWlThQeNbJ8G5D1A-t!pYmUjRgOcL9I6E3B+y(v%r#oXlTiQeNbK8G5D2A-x*t$qZnVkShPdMaI7F4C0z)w&s!pXmUjRfOcL9H6E2B+y(u%r#oWlTiQeNbJ8G5D1A-x*t$qYnVkSgPdMaI7F3C0z)v&s!pXmUiRfOcK9H6E2B+x(u%rZoWlThQeMbJ8G4D1A-w*t!qYnVjSgPdLaI7F3C0y)v&s#pXmUiRfNcK9H5E2B+x(u$rZoWkThQeMbJ7G4D1z-w*t!qYmVjSgOdLaI6F3B0y)v%s#pXlUiQfNcK8H5E2A+x*u$rZnWkThPeMbJ7G4C1z-w&t!qYmVjRgOdL9I6F3B0y(v%s#oXlUiQfNbK8H5D2A+x*u$qZnWkShPeMaJ7F4C1z)w&t!pYmUjRgOcL9I6E3B0y(v%r#oXlTiQfNbK8G5D2A-x*u$qZnVkShPdMaJ7F4C0z)w&s!pYmUjRfOcL9H6E3B+y(u%r#oWlTiQeNbJ8G5D1A-x*t$qZnVkSgPdMaI7F4C0z)v&s!pXmUjRfOcK9H6E2B+y(u%rZoWlThQeNbJ8G4D1A-w*t$qYnVjSgPdLaI7F3C0y)v&s#pXmUiRfNcK9H5E2B+x(u%rZoWkThQeMbJ8G4D1z-w*t!qYnVjSgOdLaI6F3C0y)v%s#pXlUiRfNcK8H5E2A+x(u$rZnWkThPeMbJ7G4C1z-w&t!qYmVjSgOdL9I6F3B0y)v%s#oXlUiQfNcK8H5D2A+x*u$rZnWkShPeMaJ7G4C1z)w&t!pYmVjRgOcL9I6E3B0y(v%r#oXlTiQfNbK8G5D2A-x*u$qZnWkShPdMaJ7F4C1z)w&s!pYmUjRgOcL9H6E3B+y(v%r#oWlTiQeNbK8G5D1A-x*t$qZnVkSgPdMaI7F4C0z)v&s!pXmUjRfOcL9H6E2B+y(u%r#oWlThQeNbJ8G5D1A-w*t$qYnVkSgPdLaI7F3C0z)#oWlTiQeNbK8G5D1A-x*t$qZnVkSgPdMaI7F4C0z)w&s!pXmUjRfOcL9H6E2B+y(u%r#oWlThQeNbJ8G5D1A-w*t$qYnVkSgPdLaI7F3C0z)v&s#pXmUiRfOcK9H5E2B+x(u%rZoWlThQeMbJ8G4D1A-w*t!qYnVjSgPdLaI6F3C0y)v&s#pXlUiRfNcK9H5E2A+x(u$rZoWkThPeMbJ7G4D1z-w&t!qYmVjSgOdLaI6F3B0y)v%s#pXlUiQfNcK8H5E2A+x*u$rZnWkThPeMaJ7G4C1z-w&t!pYmVjRgOdL9I6E3B0y(v%s#oXlTiQfNbK8H5D2A-x*u$qZnWkShPeMaJ7F4C1z)w&t!pYmUjRgOcL9I6E3B+y(v%r#oXlTiQeNbK8G5D2A-x*t$qZnVkShPdMaI7F4C0z)w&s!pXmUjRfOcL9H6E3B+y(u%r#oWlTiQeNbJ8G5D1A-x*t$qYnVkSgPdMaI7F3C0z)v&s!pXmUiRfOcK9H6E2B+x(u%rZoWlThQeMbJ8G4D1A-s!pYmUjRfOcL9H6E3B+y(u%r#oWlTiQeNbJ8G5D1A-x*t$qYnVkSgPdMaI7F3C0z)v&s!pXmUiRfOcK9H6E2B+x(u%rZoWlThQeNbJ8G4D1A-w*t$