信息论第二章答案(2)

H(XYZ)?????p(xiyjzk)logp(xiyjzk)ijk128827271616?12 ???log?log?log?log?103103103103103103103103881515551212? ?log?log?log?log?103103103103103103103103? ?2.836 bit/symbol H(YZ)????p(yjzk)logp(yjzk)jk20232332322828??20 ???log?log?log?log??103103103103103103103103? ?1.977 bit/symbolH(X/YZ)?H(XYZ)?H(YZ)?2.836?1.977?0.859 bit/symbol(3)

I(X;YZ)?H(X)?H(X/YZ)?0.964?0.859?0.159 bit/symbol

2.11 有两个二元随机变量X和Y，它们的联合概率为

Y X y1=0 y2=1 x1=0 1/8 3/8 x2=1 3/8 1/8 并定义另一随机变量Z = XY（一般乘积），试计算： (1) H(X), H(Y), H(Z), H(XZ), H(YZ)和H(XYZ)；

(2) H(X/Y), H(Y/X), H(X/Z), H(Z/X), H(Y/Z), H(Z/Y), H(X/YZ), H(Y/XZ)和H(Z/XY)； (3) I(X;Y), I(X;Z), I(Y;Z), I(X;Y/Z), I(Y;Z/X)和I(X;Z/Y)。

解： (1)

131p(x1)?p(x1y1)?p(x1y2)???882311p(x2)?p(x2y1)?p(x2y2)???882H(X)???p(xi)logp(xi)?1 bit/symboli131p(y1)?p(x1y1)?p(x2y1)???882311p(y2)?p(x1y2)?p(x2y2)???882H(Y)???p(yj)logp(yj)?1 bit/symbolj

Z = XY的概率分布如下：

z?0z2?1??Z???1??71???P(Z)????8??8?

2711??7H(Z)???p(zk)???log?log??0.544 bit/symbol888??8k · 6 ·

p(x1)?p(x1z1)?p(x1z2)p(x1z2)?0p(x1z1)?p(x1)?0.5p(z1)?p(x1z1)?p(x2z1)p(x2z1)?p(z1)?p(x1z1)?p(z2)?p(x1z2)?p(x2z2)p(x2z2)?p(z2)?1873?0.5?88

H(XZ)????p(x?113311?izk)logp(xizk)???log?logik?2288?8log8???1.406 bit/symbolp(y1)?p(y1z1)?p(y1z2)p(y1z2)?0p(y1z1)?p(y1)?0.5p(z1)?p(y1z1)?p(y2z1)p(y72z1)?p(z1)?p(y1z1)?8?0.5?38

p(z2)?p(y1z2)?p(y2z2)p(y2z2)?p(z2)?18H(YZ)????p(ylogp(y?113311?jzk)jzk)???log?log?log??jk?228888?1.406 bit/symbolp(x1y1z2)?0p(x1y2z2)?0p(x2y1z2)?0p(x1y1z1)?p(x1y1z2)?p(x1y1)p(x1y1z1)?p(x1y1)?1/8p(x1y2z1)?p(x1y1z1)?p(x1z1)p(x11y2z1)?p(x1z1)?p(x1y1z1)?2?18?38p(x2y1z1)?p(x2y1z2)?p(x2y1)p(x2y1z1)?p(x2y1)?38p(x2y2z1)?0p(x2y2z1)?p(x2y2z2)?p(x2y2)p(xy122z2)?p(x2y2)?8H(XYZ)?????p(xiyjzk)log2p(xiyjzk)ijk?1333311?

???1?8log8?8log8?8log8?8log8???1.811 bit/symbol(2)

7 ··

1333311??1H(XY)????p(xiyj)log2p(xiyj)????log?log?log?log??1.811 bit/symbol88888888??ijH(X/Y)?H(XY)?H(Y)?1.811?1?0.811 bit/symbolH(Y/X)?H(XY)?H(X)?1.811?1?0.811 bit/symbolH(X/Z)?H(XZ)?H(Z)?1.406?0.544?0.862 bit/symbol H(Z/X)?H(XZ)?H(X)?1.406?1?0.406 bit/symbolH(Y/Z)?H(YZ)?H(Z)?1.406?0.544?0.862 bit/symbolH(Z/Y)?H(YZ)?H(Y)?1.406?1?0.406 bit/symbolH(X/YZ)?H(XYZ)?H(YZ)?1.811?1.406?0.405 bit/symbolH(Y/XZ)?H(XYZ)?H(XZ)?1.811?1.406?0.405 bit/symbolH(Z/XY)?H(XYZ)?H(XY)?1.811?1.811?0 bit/symbol(3)

I(X;Y)?H(X)?H(X/Y)?1?0.811?0.189 bit/symbolI(X;Z)?H(X)?H(X/Z)?1?0.862?0.138 bit/symbolI(Y;Z)?H(Y)?H(Y/Z)?1?0.862?0.138 bit/symbol

I(X;Y/Z)?H(X/Z)?H(X/YZ)?0.862?0.405?0.457 bit/symbolI(Y;Z/X)?H(Y/X)?H(Y/XZ)?0.862?0.405?0.457 bit/symbolI(X;Z/Y)?H(X/Y)?H(X/YZ)?0.811?0.405?0.406 bit/symbol

2.12 有两个随机变量X和Y，其和为Z = X + Y（一般加法），若X和Y相互独立，求证：H(X) ≤ H(Z), H(Y) ≤ H(Z)。

证明：

?Z?X?Y?p(yj) (zk?xi)?Y?p(zk/xi)?p(zk?xi)?? (zk?xi)?Y?0 ??H(Z/X)????p(xizk)logp(zk/xi)???p(xi)??p(zk/xi)logp(zk/xi)?iki?k?

?? ???p(xi)??p(yj)log2p(yj)??H(Y)i?j??H(Z)?H(Z/X)?H(Z)?H(Y)同理可得H(Z)?H(X)。

2.13 设有一个信源，它产生0，1序列的信息。它在任意时间而且不论以前发生过什么符号，均按P(0) = 0.4，P(1) = 0.6的概率发出符号。 (1) 试问这个信源是否是平稳的？ (2) 试计算H(X2), H(X3/X1X2)及H∞；

(3) 试计算H(X4)并写出X4信源中可能有的所有符号。

解： (1)

这个信源是平稳无记忆信源。因为有这些词语：“它在任意时间而且不论以前发生过什么符号……” ．．．．．．．．．．．．．．．

· 8 ·

(2)

H(X2)?2H(X)??2?(0.4log0.4?0.6log0.6)?1.942 bit/symbolH(X3/X1X2)?H(X3)???p(xi)logp(xi)??(0.4log0.4?0.6log0.6)?0.971 bit/symbol

iH??limH(XN/X1X2...XN?1)?H(XN)?0.971 bit/symbolN???(3)

H(X4)?4H(X)??4?(0.4log0.4?0.6log0.6)?3.884 bit/symbolX4的所有符号：00000001001000110100010101100111100010011010101111001101111011112.14 设X?X1X2...XN是平稳离散有记忆信源，试证明：

H(X1X2...XN)?H(X1)?H(X2/X1)?H(X3/X1X2)?...?H(XN/X1X2...XN?1)。

证明：

H(X1X2...XN)????...?p(xi1xi2...xiN)logp(xi1xi2...xiN)i1i2iN????...?p(xi1xi2...xiN)logp(xi1)p(xi2/xi1)...p(xiN/xi1...xiN?1)i1i2iN?????????...?p(xi1xi2...xiN)?logp(xi1)????...?p(xi1xi2...xiN)?logp(xi2/xi1)i1?i2iNi1i2?iN?? ...???...?p(xi1xi2...xiN)logp(xiN/xi1...xiN?1)i1i2iN???p(xi1)logp(xi1)???p(xi1xi2)logp(xi2/xi1)i1i1i2 ...???...?p(xi1xi2...xiN)logp(xiN/xi1...xiN?1)i1i2iN?H(X1)?H(X2/X1)?H(X3/X1X2)?...?H(XN/X1X2...XN?1) …… 此处隐藏：1899字，全部文档内容请下载后查看。喜欢就下载吧 ……