信息论第二章答案(4)

H?????p(ei)p(ej/ei)logp(ej/ei)ij3311?1 ???p(e1/e1)logp(e1/e1)?p(e2/e1)logp(e2/e1)?p(e3/e1)logp(e3/e1)33?3111 ?p(e1/e2)logp(e1/e2)?p(e2/e2)logp(e2/e2)?p(e3/e2)logp(e3/e2) 333111? ?p(e1/e3)logp(e1/e3)?p(e2/e3)logp(e2/e3)?p(e3/e3)logp(e3/e3)?333?11111?1? ????p?logp??plogp??p?logp??p?logp??p?logp??p?logp?33333?3? ??p?logp?p?logp bit/symbol

??2.17黑白气象传真图的消息只有黑色和白色两种，即信源X={黑，白}。设黑色出现的概率为

P(黑) = 0.3，白色出现的概率为P(白) = 0.7。

(1) 假设图上黑白消息出现前后没有关联，求熵H(X)； (2) 假设消息前后有关联，其依赖关系为P(白/白) = 0.9，P(黑/白) = 0.1，P(白/黑) = 0.2，P(黑/黑) = 0.8，求此一阶马尔可夫信源的熵H2(X);

(3) 分别求上述两种信源的剩余度，比较H(X)和H2(X)的大小，并说明其物理含义。

解： (1)

H(X)???p(xi)logp(xi)??(0.3log0.3?0.7log0.7)?0.881 bit/symbol

i(2)

?p(e1)?p(e1)p(e1/e1)?p(e2)p(e1/e2)??p(e2)?p(e2)p(e2/e2)?p(e1)p(e2/e1)?p(e1)?0.8p(e1)?0.1p(e2)??p(e2)?0.9p(e2)?0.2p(e1)?p(e2)?2p(e1)??p(e1)?p(e2)?1?p(e1)?1/3??p(e2)?2/3H?????p(ei)p(ej/ei)logp(ej/ei)ijp(黑/黑)=0.8黑e1p(白/黑)=0.2p(白/白)=0.1白e2 p(白/白)=0.9122?1? ????0.8log0.8??0.2log0.2??0.1log0.1??0.9log0.9?333?3? ?0.553 bit/symbol(3)

?1?H0?H?log2?0.881??11.9%H0log2H?H?log2?0.553?1?0??44.7%H0log2H(X) > H2(X)

· 11 ·

表示的物理含义是：无记忆信源的不确定度大于有记忆信源的不确定度，有记忆信源的结构化信息较多，能够进行较大程度的压缩。

2.18 每帧电视图像可以认为是由3?105个像素组成的，所有像素均是独立变化，且每像素又取128个不同的亮度电平，并设亮度电平是等概出现，问每帧图像含有多少信息量？若有一个广播员，在约10000个汉字中选出1000个汉字来口述此电视图像，试问广播员描述此图像所广播的信息量是多少（假设汉字字汇是等概率分布，并彼此无依赖）？若要恰当的描述此图像，广播员在口述中至少需要多少汉字？

解： 1)

H(X)?logn?log128?7 bit/symbolN5

H(X)?NH(X)?3?10?7?2.1?10 bit/symbol6 2)

H(X)?logn?log10000?13.288 bit/symbolN H(X)?NH(X)?1000?13.288?13288 bit/symbol 3)

H(XN)2.1?106 N???158037H(X)13.288

· 12 ·

1??x2.19 给定声音样值X的概率密度为拉普拉斯分布p(x)??e,???x???，求Hc(X)，并

2证明它小于同样方差的正态变量的连续熵。

解：

1Hc(X)???p(x)logp(x)dx???p(x)log?e??|x|dx????2???? ??log22?2?????p(x)dx??p(x)loge??|x|dx???? ?log????1??|x|?eloge??|x|dx??2???? ?log???e??xloge??xdx0其中：???0?e??xloge??xdx????loge??xde??x0????????x?????e??xdloge??x???e??log2e?log2e000??22e?Hc(X)?log?log2e?log bit/symbol?e??xlog2e??x????01??11??|x|?exdx???e?xxdx???e??xxdx????2??202010101??1???e?xxdx???e?(?y)(?y)d(?y)???e??yydy????e??yydy??2??2??202??1??1?m????e??xxdx???e??xxdx?00202m?E(X)??p(x)?xdx????????1??|x|2?exdx???e??xx2dx????20??????????x2????x2???x2??x ???x2de??x???ex?edx?edx?2exdx?????00000??2??2??x??????x?2 ???xde??x????ex0??0edx??20?????122e?Hc(X正态)?log2?e?2?log?e?Hc(X)?log2??

?2?E?x?m?2?E(x2)??p(x)?x2dx????????

?1?2.20 连续随机变量X和Y的联合概率密度为：p(x,y)???r2??0x2?y2?r2其他，求H(X), H(Y), H(XYZ)和I(X;Y)。 ?（提示：?2log2sinxdx??0?2log22）

解：

· 13 ·

p(x)??r2?x2?r2?x2rp(xy)dy??r2?x2?12r2?x2dy? (?r?x?r)2r2?x2?r2?rHc(X)???p(x)logp(x)dx?rr2r2?x2 ???p(x)logdx2?r?rrr2 ???p(x)log2dx??p(x)logr2?x2dx?r?r?rr?r2 ?log??p(x)logr2?x2dx?r2?r21 ?log?logr?1?log2e221 ?log2?r?log2e bit/symbol2其中：?r?rp(x)logr2?x2dxr2r2?x222??logr?xdx?r?r24r2?2?r?x2logr2?x2dx?r040令x?rcos?2??rsin?logrsin?d(rcos?)?r240??2??r2sin2?logrsin?d??r2??4??4??20sin2?logrsin?d?sin?logrd???2?204??4??20sin2?logsin?d??1?cos2?41?cos2??logr?2d???2logsin?d?0?2?02

· 14 ·

???2?22?logr?d??2?0?logr?2cos2?d??20??0logsin?d??22??0cos2?logsin?d????logr?12?logr?0dsin2??2?(??2log22)?22??0cos2?logsin?d???logr?1?22??0cos2?logsin?d??logr?1?12log2e其中：2???20cos2?logsin?d?1??2??0logsin?dsin2???1?????sin2?logsin?02???20sin2?dlogsin???????1??log2e??202sin?cos?cossin?d???2?2?log2e?20cos?d????2log1?cos2??2e?202d????1??log2e?20d??1?log2e?20cos2?d????12loge?122?log2esin2?02??12log2e2p(y)??r?y2?r2?y2p(xy)dx??r2?y212r2?y2?r2?y2?r2dx??r2 (?r?y?r)p(y)?p(x)H(Y)?H?r?1CC(X)?log22log2e bit/symbolHc(XY)????p(xy)logp(xy)dxdyR ????p(xy)log1dxdy

R?r2 ?log?r2??p(xy)dxdyR ?log22?r bit/symbolIc(X;Y)?Hc(X)?Hc(Y)?Hc(XY) ?2log2?r?log2e?log?r2 ?log2??log2e bit/symbol

15 ·

·