What is the minimum value of H(p1,…,pn)=H(P) as P ranges over the set of n-dimensional probability v
What is the minimum value of H(p1,…,pn)=H(P) as P ranges over the set of n-dimensional probability vectors? Find all P's which achieve the minimum.
根据离散随机变量X的信息熵H(X)的性质:H(X)≥0,所以熵函数
H(P1,…,Pn)=H(P)≥0
其中P是n维概率矢量。因此,熵函数的最小值等于0。
要使H(p1,…,Pn)=0,n维概率矢量P=(p1,…,Pn)必须有一个概率分量为1,其他概率分量均为0,所以,使熵函数达到最小值的n维概率矢量JP可以是:(1,0,…,0,0),(0,1,…,0,0),…,(0,0,…,0,1)。