信息论第二章答案(5)

2.21 设X?X1X2...XN是N维高斯分布的连续信源，且X1, X2, … , XN的方差分别是

22，它们之间的相关系数?(XiXj)?0(i,j?1,2...,N,i?j)。试证明：N维高斯分布?12,?2,...,?N的连续信源熵

1NHc(X)?Hc(X1X2...XN)??log2?e?i2

2i证明：

相关系数?xixj?0 ?i,j?1,2,...,N, i?j?，说明X1X2...XN是相互独立的。

???Hc(X)?Hc(X1X2...XN)?Hc(X1)?Hc(X2)?...?Hc(XN)1?Hc(Xi)?log2?e?i22?Hc(X)?Hc(X1)?Hc(X2)?...?Hc(XN)11122 ?log2?e?12?log2?e?2?...?log2?e?N2221N ??log2?e?i22i?1?bx22.22 设有一连续随机变量，其概率密度函数p(x)???00?x?a其他

(1) 试求信源X的熵Hc(X)；

(2) 试求Y = X + A (A > 0)的熵Hc(Y)； (3) 试求Y = 2X的熵Hc(Y)。

解： 1)

Hc(X)???f(x)logf(x)dx???f(x)logbx2dxRR ??logb??f(x)dx??f(x)logx2dxRR ??logb?2b?x2logxdxR2ba3a3 ??logb?log9ebx3ba3?FX(x)?,FX(a)??1332a3?Hc(X)??logb??log bit/symbol3e2)

· 16 ·

?0?x?a?0?y?A?a?A?y?a?AFY(y)?P(Y?y)?P(X?A?y)?P(X?y?A) ??y?AAbx2dx?b3(y?A)3f(y)?F?(y)?b(y?A)2

Hc(Y)???Rf(y)logf(y)dy???Rf(y)logb(y?A)2dy ??logb??f(y)dy??f(y)log(y?A)2RRdy ??logb?2b?(y?A)2Rlog(y?A)d(y?A) ??logb?2ba39loga3 e bit/symbolbba3?F3Y(y)?3(y?A),FY(a?A)?3?1

?HY)??logb?23?loga3c(e bit/symbol3)

?0?x?a?0?y2?a?0?y?2aFyY(y)?P(Y?y)?P(2X?y)?P(X?2)y ??2b30bx2dx?24yf(y)?F?(y)?by28H???bc(Y)Rf(y)logf(y)dy???Rf(y)log8y2dy ??logb8??Rf(y)dy??Rf(y)logy2dy ??logb8?b4?Ry2logydylogb2ba38a3 ??8?9loge ??logb?2ba3a39?2ba39loge?3b3?F3ba Y(y)?24y,FY(2a)?3?1?logb?2a3?Hc(Y)?3?loge?1 bit/symbol

17 ··